The Importance of Net Promoter Score (NPS) Surveys in Building Products and the Rise of Llama 2: The New Open LLM SOTA

The Importance of Net Promoter Score (NPS) Surveys in Building Products and the Rise of Llama 2: The New Open LLM SOTA

Introduction:

When it comes to building products, businesses often rely on various tools and metrics to gauge their success. One such tool is Net Promoter Score (NPS) surveys, which have gained popularity in recent years. However, the effectiveness of NPS surveys has been a topic of debate. In this article, we will explore the role of NPS surveys in product development, while also delving into the emergence of Llama 2, a new open-source model that is revolutionizing the field.

Net Promoter Score (NPS) Surveys: A Directional Compass

NPS surveys have been hailed as a valuable tool in understanding customer sentiment and guiding product development. The concept behind NPS is simple - it measures the likelihood of customers recommending a product or service to others on a scale of 0 to 10. However, it is important to note that NPS surveys do not provide a complete picture of customer satisfaction.

The Limitations of NPS Surveys

While NPS surveys offer a quantitative representation of qualitative data, they should not be the sole basis for decision-making. NPS scores can be influenced by various factors, such as cultural differences in grading or the context in which the survey is conducted. For example, Netflix experienced high NPS scores in South America, but poor customer retention during the initial months. This highlights the need to consider other metrics, such as monthly customer retention, as a proxy for product quality.

The Role of NPS in the Survey Toolkit

NPS surveys are just one tool in the survey toolkit for understanding customers. Demographics, competitive product usage, and entertainment preferences are all important aspects to capture. NPS provides a simple and easy-to-understand signal, but it should be complemented by other data sources to form a comprehensive understanding of the customer.

The Importance of Stories and Examples

In the realm of product development, stories and examples play a crucial role in bringing ideas to life. While NPS surveys provide numerical data, the qualitative feedback from customers is equally valuable. The "What's good?/What could be better?" verbatims derived from NPS surveys can provide valuable insights and help form hypotheses for future experiments.

Actionable Advice:

1. Use NPS as a directional compass: While NPS surveys may not present a complete story, they can serve as a guide in understanding customer sentiment. Use NPS scores as a starting point and complement them with other metrics for a holistic view.

2. Consider other proxy metrics for product quality: Monthly customer retention can often be a better indicator of product quality than NPS scores. Analyze customer behavior and actions rather than solely relying on what they say in surveys.

3. Embrace a diverse range of data sources: While NPS surveys are valuable, they should be supplemented with other data sources, such as demographics, usage patterns, and qualitative feedback. This will provide a more comprehensive understanding of the customer and inform product development strategies.

The Rise of Llama 2: The New Open LLM SOTA

While NPS surveys have their own set of limitations, the field of product development has witnessed the emergence of a new open-source model called Llama 2. Llama 2 has been praised for outperforming other open-source models in its weight class. This has sparked interest in industries, such as healthcare, government, and military organizations, where sensitive data cannot be shared with external API providers.

The Need for Open Models

Open models like Llama 2 have gained significance due to the need for organizations to have total control over their data. Businesses and government entities that deal with sensitive information cannot afford to rely on external API providers, even if they are trusted names like OpenAI through Azure. By owning and serving their own models, these organizations can ensure the security and privacy of their data.

Conclusion:

Net Promoter Score (NPS) surveys have been a valuable tool in product development, providing insights into customer sentiment. However, it is important to recognize their limitations and complement them with other metrics and data sources. Additionally, the rise of open-source models like Llama 2 offers new possibilities for organizations that require total control over their data. By embracing a diverse range of tools and approaches, businesses can enhance their product development strategies and better cater to customer needs.