The New Language Model Stack: Building the Future of AI

The New Language Model Stack: Building the Future of AI

In recent years, the integration of language models into various products has become a common practice for companies in the Sequoia network. A staggering 65% of these companies already have language model applications in production, a significant increase from 50% just two months ago. The remaining companies are still experimenting with the possibilities that language models offer.

When it comes to choosing a language model, OpenAI's GPT is the clear favorite, with 91% of companies in the sample using it. However, there has been a growing interest in Anthropic, which saw a rise in popularity over the last quarter, reaching 15%. This indicates that companies are actively exploring different options and seeking the best fit for their needs.

One key aspect that 88% of companies agreed on is the importance of a retrieval mechanism, such as a vector database, as part of their language model stack. Retrieving relevant context for a model to reason about greatly enhances the quality of results, reduces inaccuracies, and solves data freshness issues. This retrieval mechanism is essential for companies to customize language models to their unique contexts, making them more powerful and effective.

Customizing language models can be approached in different ways. One option is to train a custom model from scratch, which offers the highest degree of customization but also comes with a medium level of difficulty. Fine-tuning a base model is another approach, but it can be challenging and may have unintended consequences such as model drift or the model "breaking" in unexpected ways. Currently, this approach is out of reach for most companies. Finally, using a pre-trained model and retrieving relevant context is the lowest difficulty option. This approach makes unstructured data easily searchable using natural language and can be seamlessly integrated into existing services.

While the stack for language model APIs and custom model training may seem separate at the moment, they are gradually converging. As AI interest grows and open-source development accelerates, more companies are becoming interested in training and fine-tuning their own models. This convergence will lead to a more unified stack that caters to both API users and those looking to train their own models.

The good news for developers is that the language model stack is becoming increasingly developer-friendly. Tools like LangChain abstract away common problems, making it easier to build language model applications. These tools help combine models into higher-level systems, chain multiple calls to models, connect models to tools and data sources, and build agents that can operate those tools. This developer-friendly approach also helps avoid vendor lock-in, making it easier to switch between different language models.

Despite the advancements in language models, there are still challenges that need to be addressed for full adoption. Language models need to become more trustworthy in terms of output quality, data privacy, and security. These improvements are crucial for gaining widespread trust and acceptance of AI-powered language model applications.

Looking ahead, language model applications will become increasingly multi-modal. AI is permeating every aspect of technology, and only 65% of surveyed companies currently have language model applications in production. This indicates that there is still immense potential for growth and innovation in this field.

To make the most of language models and compound their benefits, here are three actionable pieces of advice:

1. Invest in compound time: Just like successful people who spend 10 hours a week on "compound time," it is essential to allocate dedicated time for activities that have a long-term payoff. Slow down, work less, and prioritize learning and thinking, as these activities will yield significant returns over time.

2. Embrace different modes of learning: Walking, napping, and reading are all activities that can enhance learning, memory, creativity, and productivity. Take 15 minutes each day to go for a walk, incorporate power naps into your routine, and make reading a regular habit. These simple practices can have a profound impact on your overall performance.

3. Foster social connections and experimentation: Engaging in conversations with others and seeking collaboration can lead to surprising breakthroughs. As Shonda Rhimes discovered in her Year of Yes experiment, saying yes to things that scare you can open doors to new opportunities. Additionally, adopting a habit of experimentation, like Peter Drucker, can help you learn from experience and make better decisions.

In conclusion, the new language model stack is revolutionizing the way companies leverage AI capabilities. With the increasing adoption of language models, the convergence of different stacks, and the focus on developer-friendliness, the future looks promising. By addressing challenges related to trustworthiness and embracing multi-modal applications, language models have the potential to transform various industries and empower businesses to achieve their goals. So, embrace the possibilities of language models and make the most of compound time for long-term success.