The Importance of Net Promoter Score (NPS) Surveys in Product Building

The Importance of Net Promoter Score (NPS) Surveys in Product Building

Net Promoter Score (NPS) surveys have become a popular tool for gathering feedback and measuring customer satisfaction. However, their effectiveness in building products has been a topic of debate. While some organizations swear by NPS as a "single source of truth," others find it lacking in providing a complete picture. In fact, only 31% of respondents in a survey found NPS surveys helpful. So, what's the deal with NPS?

NPS is often treated as a compass, providing a directional indicator of customer sentiment. It presents a single number that seems quantitative, but in reality, it's a numerical representation of qualitative data. Therefore, relying solely on NPS can be misleading. To truly understand customers and their preferences, a more comprehensive approach is needed.

Surveys are an essential tool in capturing customer insights. They help us understand who our customers are and how they think. This includes gathering demographic information, analyzing competitive product usage, and even delving into their entertainment preferences. NPS is just one part of this survey toolkit, providing a signal that is easy to interpret. The verbatims accompanying NPS responses, such as "What's good?" and "What could be better?", can be incredibly helpful in identifying areas for improvement.

However, NPS should not be the sole metric for determining product quality. At Netflix, for example, NPS data was collected but rarely looked at. Instead, they found monthly customer retention to be a much better proxy metric for product quality. Retention measures what customers do, rather than what they say. Similarly, at Chegg, an NPS survey yielded a high score of 60, indicating customer satisfaction. However, poor retention rates in the following months revealed a different reality. It turns out that Brazilians, for example, tend to be more generous in their ratings compared to Australians or Germans.

It's important to note that all data sources, except for NPS, can be manipulated or gamed. By aggressively asking for positive feedback, one can easily increase NPS scores. Therefore, it's crucial to take NPS numbers with a grain of salt and dig deeper into the qualitative data to form hypotheses for future experiments. NPS is just a number, and it's the stories and examples that truly bring ideas to life.

The End of Organizing: Unlocking the Power of Language Models

In the age of advanced technology, organizing our notes may become a thing of the past. Large language models (LLMs) like GPT-3 have the potential to revolutionize how we access and utilize our old notes. Instead of spending time organizing and categorizing our notes, we can rely on AI intelligence to surface the right note at the right time and in the right format for optimal effectiveness.

The challenge with traditional note-taking is that we often jot down information without knowing exactly how or when we will use it in the future. We treat note-taking as insurance, ensuring that we capture important details for later reference. However, when we revisit these old notes, we must mentally recreate the context in which they were taken to understand their relevance to our current tasks. This process can be time-consuming and inefficient.

LLMs have the potential to eliminate the need for manual organization. These models can create an automated taxonomy of our notes, making it easier for us to navigate through them. Beyond simple tagging and linking, LLMs can understand the context of our notes and present them to us in a way that instantly clicks with our current work. Imagine having a summary of key relationships or patterns in your thinking over time, generated by an LLM. This could provide a valuable history of your mind on a specific topic, helping you understand yourself and the world better.

The true power of LLMs lies in their ability to turn our notes into a second brain. They can enrich our notes as we write them, providing additional context and insights. They can automatically categorize and synthesize our notes, making them more accessible and usable in the future. In essence, LLMs can serve as the ultimate tool for thought, providing us with the information we need, when and where we need it.

Three Actionable Advice for Building Better Products and Leveraging LLMs:

1. Embrace a holistic approach: While NPS surveys can provide valuable insights, they should not be the sole basis for decision-making. Look beyond NPS and consider metrics like customer retention to gain a comprehensive understanding of product quality.

2. Utilize qualitative data: Don't overlook the power of qualitative data. Verbatims from NPS surveys and other sources can provide rich context and help form hypotheses for experimentation. Combine quantitative and qualitative data to gain a more well-rounded perspective.

3. Embrace AI intelligence: As technology advances, leverage tools like LLMs to unlock the power of your notes. Let AI do the organizing for you, allowing you to focus on deriving insights and utilizing your notes effectively. Embrace the potential of LLMs to create a second brain and enhance your thinking process.

In conclusion, NPS surveys can be a helpful tool in building products, but they should not be relied upon as the sole source of truth. A holistic approach that combines qualitative and quantitative data is essential for understanding customer preferences and improving product quality. Additionally, as technology evolves, AI intelligence, such as LLMs, can revolutionize how we access and utilize our notes, making organization a thing of the past. By embracing these approaches and leveraging the power of technology, we can build better products and enhance our thinking processes.