Examining Emergent Abilities in Large Language Models: Insights from Pinterest's Retention Wins

Examining Emergent Abilities in Large Language Models: Insights from Pinterest's Retention Wins

The concept of emergence, popularized by Nobel laureate Philip Anderson in his essay "More is Different," suggests that quantitative changes in a system can lead to new behaviors. This phenomenon has been observed in various disciplines, including physics, biology, economics, and computer science. In the context of large language models, emergence refers to the development of new abilities that are not present in smaller models but emerge as the model scales up. These emergent abilities are of scientific interest and drive further research in the field of large language models.

Pinterest, a platform that CEO Ben Silbermann describes as "more of a personal tool than a social one," offers unique insights into the relationship between product engagement and revenue generation. Unlike traditional social media platforms, Pinterest focuses on connecting individuals with content and ideas that they discover. This distinction allows for a natural integration of advertising that enhances the user experience rather than disrupting it. The key metric for measuring engagement on Pinterest is the "Save" action, which indicates that a user has found something of interest. This metric not only signals user satisfaction but also provides valuable insights into the quality of content on the platform.

When it comes to retention and growth, Casey Winters, an expert in growth strategy, emphasizes the importance of aligning metrics with the company's mission. Mission-oriented companies, like Pinterest, prioritize metrics that indicate fulfillment of their mission, while business-oriented companies prioritize metrics that drive long-term revenue. Determining the right retention strategy involves measuring the desired action within a natural time frame. In the early stages of a startup, targeting the normal frequency of a behavior can be effective, while in the long term, increasing the frequency may be necessary.

To achieve significant growth, Winters advises conducting a variety of experiments, including those that challenge industry "best practices." He highlights the importance of learning from both successful and failed experiments. For example, a simple experiment on Pinterest that replaced the default scrolling behavior with a call-to-action banner resulted in a significant improvement in conversion rates. On the other hand, an experiment involving emails with relevant content failed to improve activation rates. These insights helped the team focus on what truly mattered to users and refine their growth strategies accordingly.

While growth teams often rely on emails and notifications to improve retention, Winters acknowledges the limitations of these tools. To overcome this challenge, he suggests exploring product changes that can influence retention. In one instance, GrubHub introduced a game that rewarded users with free food after every third order. This creative approach increased order frequency and sustained user engagement. By thinking beyond traditional tactics and channels, companies can uncover new opportunities for growth.

Pinterest's experience with SEO experiments further emphasizes the importance of questioning established best practices. Instead of blindly following industry norms, companies should analyze the strengths and weaknesses of their business and adapt their strategies accordingly. Understanding the algorithms and building products that align with both user needs and search engine criteria can unlock the full potential of a channel.

In conclusion, examining emergent abilities in large language models offers valuable insights into the potential of scaling up. Pinterest's retention wins provide unique perspectives on aligning metrics with the company's mission and challenging conventional growth strategies. To apply these insights to your own business, consider the following actionable advice:

1. Align metrics with your company's mission: Identify the key metrics that indicate fulfillment of your mission, rather than focusing solely on revenue-related metrics.

2. Experiment with divergent approaches: Don't be afraid to challenge industry best practices and conduct experiments that test entirely new experiences. The biggest growth wins often come from a series of small experiments.

3. Think beyond traditional tactics and channels: Explore product changes that can address retention bottlenecks and enhance the core value of your product. Look for opportunities to integrate advertising or incentives that align with user needs.

By adopting these strategies, you can unlock the potential for emergent growth and drive success in your own business.