What The Next Generation of AI Companies Will Look Like

TL;DR

The next generation of AI companies should choose APIs, open-source systems, fine-tuned models, or custom large models according to the compounding loop and differentiation they want to build. APIs make it possible to prototype in an afternoon, test product-market fit, and identify the right problem, while greater reliability or durability may require more control. Read on for a practical framework for navigating these choices without prematurely pre-training a 500-billion-parameter model.

Transcript

I think the general thing here is that thinking about interfaces with the differentiation that you want to have as a business is going to be really key because I think the world that I don't think we want to live in is one where effectively these companies become sort of outsourced customer Discovery engines and then new Amazon Basics versions of t... Read More

Key Insights

• 🤗 The decision of whether to build on an open AI API, open source, or create a large model is crucial for startups and depends on factors such as differentiation, the compounding loop, and needed reliability.
• 🚒 Companies should be cautious not to become outsourced customer discovery engines, where competitors can replicate their offerings easily.
• 🥺 APIs offer a quick way to prototype and explore AI possibilities, but investing in fine-tuning and custom models can lead to better performance.

Questions & Answers

Q: How should a founder choose between an AI API, open source, and building a large model?

First determine the loop the company will run to compound its growth, such as deeply understanding a particular customer use case or building a data flywheel. Then choose the technical approach that supports the company’s intended differentiation and required level of reliability.

Q: What will differentiate the next generation of AI companies?

Their differentiation should connect directly to a compounding company loop, such as deeper understanding of a specific customer use case or a data flywheel. Without that connection, a company risks acting as an outsourced customer-discovery engine whose offering can later be replicated by an “Amazon Basics” version.

Q: Why is a company’s compounding loop the most important first consideration?

The compounding loop defines how the company’s advantage strengthens over time. Identifying whether that loop centers on customer understanding, data, or another source of differentiation helps determine where and how the company should consume or build AI services.

Q: How should reliability affect the choice of AI infrastructure?

Founders should ask how many “nines” of reliability their customers require. If the product needs many nines, consuming AI through an intermediary can be difficult because the company may lack the affordances needed to deliver the required customer experience.

Q: What are the advantages of using an AI API for prototyping?

An API makes it extremely easy to begin experimenting: a founder can sit down in an afternoon, enter some data, and get a sense of what is possible. This supports faster iteration and helps reduce product-market-fit risk before making a larger model investment.

Q: Is initial API performance the best an AI product can achieve?

Not necessarily; an afternoon API experiment can represent a lower bound on performance. Further investment, fine-tuning, and custom components may improve the result after the team has identified a promising application.

Q: How can a startup move from experiments toward a durable AI product?

A startup can begin with a human “Wizard of Oz” experiment to resolve interface questions and test whether the product makes sense. It can then replace the human with an API and later evaluate whether its data supports a fine-tuned T5 model or another smaller model.

Q: When should a startup pre-train its own very large model?

Pre-training a 500-billion-parameter model should be the last thing in mind when the startup does not yet know what application it is building. In some cases, available data and a fine-tuned T5 model or another much smaller model may already be effective.

Summary

In this video, the speaker discusses the importance of thinking about interfaces and differentiation in building a company. They explore the decision-making process for choosing between building on an open AI API, using open source tools, or creating their own large model. They emphasize the need to understand the loop that will compound the company's growth and consider the level of reliability required. They also highlight the ease of prototyping with APIs but caution against relying solely on them for long-term durability without transitioning to a more integrated solution.

Questions & Answers

Q: How should a founder navigate the decision of building on top of an open AI API, using open source tools, or creating their own large model?

The speaker suggests that figuring out the compounding loop for the company is the most important first step. They advise considering whether the company's focus will be on deeply understanding a particular customer use case or building a data flywheel. It is crucial to align this with the desired differentiation as a business to avoid becoming an outsourced customer discovery engine for others.

Q: How does the level of reliability needed impact the decision-making process?

The speaker explains that if a company requires a high level of reliability, relying on intermediaries, such as APIs, may limit the available affordances. In such cases, choosing a different approach, such as building custom solutions, becomes necessary to ensure the necessary reliability and control over the service.

Q: What advantages does using APIs offer in terms of getting started and prototyping quickly?

The speaker points out that using APIs allows for easy and quick experimentation. By investing just an afternoon, one can get a sense of the possibilities and explore potential solutions. Although the initial results may serve as a lower bound, further investment and fine-tuning can lead to substantial improvements.

Q: How can durability be built into a product if it initially relies on someone else's API?

The speaker mentions that transitioning away from using an API is a discrete process. They suggest starting with human Wizard of Oz experiments, where a human performs the tasks to identify interface issues and refine the concept. Eventually, the human can be replaced with an API to gauge its effectiveness. Depending on the specific application, a smaller, fine-tuned model may even be more effective than a large, pre-trained one.

Q: What caution does the speaker give regarding building large models without a specific application in mind?

The speaker advises against pre-training a 500 billion parameter model without a clear understanding of the application it will be used for. Instead, they emphasize the importance of identifying the right problem to solve, collecting relevant data, and iterating through experimentation and feedback.

Q: How can building on top of APIs de-risk the product-market fit?

Building on top of APIs allows for quick prototyping and testing, which helps to de-risk the product-market fit. By rapidly trying out different approaches and ideas, entrepreneurs can gather valuable feedback and iterate faster, increasing the chances of finding a reasonable solution.

Q: Is there a trade-off between quick prototyping and long-term durability in relying on APIs?

The speaker acknowledges that relying solely on APIs for long-term durability can be a concern. While APIs are excellent for initial experimentation and prototyping, there may be interface issues, dependence on external factors, or lack of customization that can limit the long-term viability of the product. Transitioning to a more integrated solution becomes necessary for ensuring durability.

Q: How can human Wizard of Oz experiments be used to refine the interface before transitioning to APIs?

The speaker explains that by using a human in the initial stages, interface issues and the overall feasibility of the concept can be identified and worked out efficiently. This allows for better understanding and optimization before replacing the human with an API-based solution. The transition is gradual, improving the overall reliability and effectiveness of the product.

Q: How does the flexibility of APIs impact the decision-making process?

The speaker mentions that APIs offer tremendous flexibility by allowing quick experimentation and proof-of-concepts. Entrepreneurs can explore various possibilities with minimal investment, which helps refine their understanding of the problem space and identify potential product-market fits. This flexibility is valuable but should be balanced with long-term durability considerations.

Q: How can the lack of affordances from intermediaries impact the level of reliability needed?

The speaker highlights that if a high level of reliability is required, relying on intermediaries, such as APIs, may limit the available affordances. By building custom solutions, companies can have better control over reliability and ensure that the specific needs of their customers are met effectively.

Takeaways

In summary, when building a company, it is crucial to consider interfaces, differentiation, and reliability. Understanding the compounding loop that will drive growth and aligning it with the company's desired differentiation is essential. APIs offer easy prototyping opportunities and de-risking of product-market fit but may lack long-term durability and customization. Gradually transitioning from human-backed experiments to API-based solutions can improve the interface and reliability. Ultimately, balancing quick prototyping with long-term durability is key.

Summary & Key Takeaways

• Companies should consider their differentiation and how it interfaces with their chosen model to avoid becoming outsourced customer discovery engines.

• The decision of whether to build on an open AI API, open source, or create a large model depends on the loop a company wants to run to compound its growth.

• The level of reliability needed should also be a factor in the decision-making process, as intermediary consumption may lack necessary affordances.