The Dangers of Vanity Metrics and Strategies for Successful New Business Product Management

The Dangers of Vanity Metrics and Strategies for Successful New Business Product Management

In the world of business, metrics play a crucial role. They provide valuable insights into the performance and growth of a company. However, not all metrics are created equal. There is a category of metrics known as vanity metrics that can be misleading and harmful to businesses. Vanity metrics are numbers that make you look good but have no material impact on your ability to make good decisions and improve your business. They are superficial and fail to drive durable revenue growth. In this article, we will explore the dangers of vanity metrics and discuss strategies for successful new business product management.

One key characteristic of a good metric is understandability. A good metric is one that’s easy for everyone to understand and track. When metrics are clear and easily comprehensible, it becomes easier for teams to align their efforts towards achieving common goals. Comparative metrics are also valuable as they allow us to compare things over periods of time to see trends. By tracking and analyzing comparative metrics, businesses can identify patterns and make informed decisions. Furthermore, if you take a comparative number and turn it into a ratio or rate, it becomes even more valuable. Ratios and rates provide deeper insights into the performance and efficiency of a business.

Now let's discuss some specific vanity metrics and their potential dangers. One such metric is the number of daily and monthly active users. While this metric may seem like a positive indicator of growth, it can be misleading if these users are not actively engaged with the core product and generating value. It is important to ensure that active users are performing actions linked to the core product's value in order for this metric to be meaningful.

Another vanity metric to be cautious of is net churn. Net churn refers to the rate at which customers are leaving a business. If you can't accurately determine how many gross customers are dropping out, you may be masking a larger problem through your client acquisition pace. It is essential to have a clear understanding of customer retention and accurately track customer churn to make informed decisions about improving customer satisfaction and loyalty.

Cumulative metrics are another type of vanity metric that can be misleading. Cumulative metrics add up all the data over a certain period of time, but they fail to provide insights into the underlying trends and patterns. It is important to analyze and interpret cumulative metrics in the context of other relevant factors to make informed decisions.

Funding raised is a metric that may give a company the opportunity to run more experiments and learn from them, but it can also lead to spending on unnecessary items. It is vital to prioritize spending and invest in areas that have a direct impact on business growth and customer satisfaction. Money can buy you time, but it can't buy you execution.

ESG scores, which measure a company's environmental and social impact, can be misleading as well. These scores are potentially even more misleading than Net Promoter Scores. It is important to critically evaluate and analyze ESG scores to ensure they provide an accurate reflection of a company's sustainability and social responsibility efforts.

For Open Source companies, GitHub stars are a widely cited vanity metric. While it may seem impressive to have a high number of stars, it is important to remember that these stars do not necessarily translate into tangible business value. Rather than focusing solely on GitHub stars, it is important to evaluate the actual impact of an Open Source project on the community and its ability to drive collaboration and innovation.

Net Dollar Retention (NDR) is a metric that can be easily manipulated and skewed. A lot of companies will game NDR by grouping it by customer size. For example, enterprise customers should naturally have high NDR, but this statistic benefits from survivorship bias. It is crucial to analyze NDR in a more granular way, taking into account different go-to-market segments and customer sizes to get a more accurate understanding of customer retention and revenue growth.

Segmentation is a key aspect of effective metric analysis. Metrics are only as useful as you actually being able to apply them to run the business. If you don't segment your data, you never actually use them. It is important to segment metrics based on different go-to-market segments of your business, which have different inputs (costs) and outputs (deal sizes). By understanding the variations across different segments, you can make informed decisions about resource allocation and optimization.

Customer Lifetime Value (LTV) is a metric that measures the value a customer brings to a business over their lifetime. However, it is essential to consider the longevity and reliability of the data used to calculate LTV. Customers need an opportunity to actually churn before you count them, and typically a track record of under three years is unreliable. It is important to rely on assumptions and projections when calculating LTV for new businesses.

Headcount is another metric that can be misleading if not analyzed in the right context. HC on its own is just a reflection of cost without any context to productivity. It is important to align headcount with business goals and evaluate the efficiency and productivity of each team member to make informed decisions about resource allocation and team optimization.

Net Promoter Score (NPS) is a metric commonly used to measure customer loyalty and satisfaction. However, it is important to critically evaluate the data used to calculate NPS. Some companies may manipulate NPS scores by taking biased reviews into account. It is crucial to ensure that NPS scores are based on reliable and unbiased data to accurately measure customer sentiment and satisfaction.

Adjusted EBITDA is a financial metric often used to evaluate a company's profitability. However, it is important to dig into the assumptions behind EBITDA calculations to ensure they are accurate and reflective of the company's true financial performance. Like for Like (LFL) revenue is another financial metric that needs to be analyzed carefully. LFL revenue can be misleading if not adjusted for inflation and foreign exchange impacts. It is crucial to consider both price and volume when evaluating revenue growth and identify any potential biases or distortions.

In conclusion, vanity metrics can be dangerous for businesses as they provide a false sense of success and fail to drive durable revenue growth. It is essential to focus on metrics that are understandable, comparative, behavior-changing, and void of segmentation. By prioritizing meaningful metrics and avoiding vanity metrics, businesses can make informed decisions and drive sustainable growth. Here are three actionable pieces of advice for successful new business product management:

1. Clearly define and track metrics that align with your business goals and provide meaningful insights into performance and growth.
2. Regularly evaluate and analyze metrics in the context of other relevant factors to gain a comprehensive understanding of your business's strengths and weaknesses.
3. Prioritize metrics that drive behavior change and enable data-driven decision-making to optimize resource allocation and drive sustainable growth.

By following these strategies and avoiding the pitfalls of vanity metrics, businesses can navigate the complexities of new business product management and set themselves up for success.