The Costly Challenges of Yelp's Comeback and AI Compute

The Costly Challenges of Yelp's Comeback and AI Compute

Introduction:
Yelp, once the go-to platform for customer reviews, is planning a comeback with reservations and pickup tools. However, it faces tough competition and the need to regain the trust of disillusioned restaurant partners. On the other hand, the high cost of AI compute poses significant challenges for the industry. The demand for compute resources in AI outstrips the supply by a factor of 10, leading to exorbitant costs for training and inference. In this article, we will explore the common points between Yelp's comeback and the expensive nature of AI infrastructure, and discuss actionable advice for businesses in both domains.

Yelp's Battle for Survival:
Yelp's struggle to regain its position as the top customer review platform while preventing further market share loss to reservation apps is a multi-pronged battle. It must address the disillusionment of restaurant partners who have been dissatisfied with its ads-based business model. To win back their trust, Yelp needs to develop transparent and effective strategies that prioritize the interests of its partners.

The Costly Reality of AI Compute:
The cost of AI compute is a predominant factor driving the industry today. Companies are spending a significant portion of their capital on compute resources, with some allocating more than 80% of their total raised capital. The supply of compute is severely constrained, resulting in a demand-supply gap of 10 times. This high cost of training and inference poses a significant barrier for businesses in the AI domain.

Understanding AI Compute Complexity:
The complexity of AI infrastructure arises from the algorithmic challenges involved in training and inference. The computational cost of AI models, such as GPT-3, is determined by the number of parameters and tokens. The rule of thumb for transformers is that a forward pass for a model with p parameters and input/output sequences of n tokens each takes approximately 2np floating point operations (FLOPs). Training a model like GPT-3 can take a staggering amount of floating point operations.

Navigating AI Infrastructure Costs:
For businesses in the AI domain, managing AI infrastructure costs is crucial for sustainability and competitiveness. Here are three actionable pieces of advice to navigate the high cost of AI compute:

1. Optimize Model Selection: Choose the smallest model that adequately solves your use case. Picking a model that matches your requirements while minimizing compute and memory consumption can significantly reduce costs. Consider the trade-off between model size, performance, and cost.

2. Leverage Hosted Model Services: Startups, especially app companies, can utilize hosted model services like OpenAI or Hugging Face. These services provide infrastructure and models, allowing rapid experimentation and product-market fit without the need to manage underlying infrastructure. This reduces costs and enables faster development.

3. Consider Cloud Infrastructure: In most cases, the cloud is the right place for AI infrastructure. Cloud providers offer compute capacity, availability, network interconnects, and customer support. The cloud eliminates the need for significant upfront hardware investments and provides scalability and flexibility. However, at very large scales, running your own data center may become more cost-effective.

Conclusion:
Yelp's comeback and the challenges of AI compute share common themes of competition, trust-building, and cost considerations. To succeed, Yelp must address the needs of both customers and restaurant partners while navigating a competitive landscape. Likewise, businesses in the AI domain must optimize model selection, leverage hosted model services, and consider cloud infrastructure to manage the high cost of AI compute. By implementing these actionable strategies, businesses can stay competitive and overcome the financial barriers of AI infrastructure.