The New Language Model Stack: Building the Future of AI Applications

The New Language Model Stack: Building the Future of AI Applications

Large language models (LLMs) have become a game-changer for companies across various industries. From code auto-complete to customer support chatbots, LLMs have revolutionized the way businesses interact with their customers and streamline their workflows. In this article, we will explore the new language model stack and how companies are leveraging LLMs to enhance their products and services.

The new stack for LLM applications revolves around language model APIs, retrieval mechanisms, and orchestration. While open source usage is also on the rise, companies are increasingly relying on foundation model APIs to integrate LLMs into their systems. OpenAI's GPT is currently the favorite choice among companies, but interest in other models like Anthropic is also growing.

One key aspect of the LLM stack is the use of retrieval mechanisms, such as vector databases. These mechanisms help provide relevant context to the LLMs, improving the quality of results and reducing inaccuracies. Many companies utilize purpose-built vector databases, while others opt for offerings from platforms like pgvector or AWS. Retrieval mechanisms are considered a crucial part of the stack by 88% of companies surveyed.

Another interesting development in the LLM stack is the adoption of orchestration and application development frameworks like LangChain. Around 38% of companies expressed interest in using such frameworks, either for prototyping or in production. The popularity of LangChain and similar tools has grown in recent months, indicating their potential for facilitating LLM integration.

While the majority of companies are focused on utilizing LLM APIs, there is a growing trend of building custom language models from scratch or open source. This approach allows companies to train models tailored to their unique contexts and data. Custom model training has seen a meaningful increase, with companies relying on compute resources, model hubs, and training frameworks provided by platforms like Hugging Face, Replicate, and PyTorch.

However, despite the rapid evolution of the LLM stack, experts agree that AI is moving too quickly to have complete confidence in the end-state stack. Nevertheless, LLM APIs are expected to remain a key pillar, followed by retrieval mechanisms and development frameworks. Open source and custom model training are also gaining traction, enabling companies to further customize their language models.

The ability to customize language models is crucial for many use cases. While generalized LLMs are powerful, they may not be sufficient or differentiating for specific industries. Companies want to enable natural language interactions on their own data, whether it's developer documentation, product inventory, or HR rules. Some companies even seek to customize models based on their users' data, such as personal notes or design layouts.

There are currently three main ways to customize language models. The most difficult method is training a custom model from scratch, which requires skilled ML scientists, relevant data, and training infrastructure. Fine-tuning a base model is a less challenging approach, involving updating pre-trained model weights with proprietary or domain-specific data. The easiest method is to use a pre-trained model and retrieve relevant context when needed. This approach leverages embeddings retrieval and allows unstructured data to be easily searchable using natural language.

While the LLM stack is still evolving, it is clear that companies are embracing the power of language models to transform their businesses. As more companies adopt LLMs, we can expect increased interest in tools to monitor LLM outputs, cost, and performance, as well as A/B test prompts. Additionally, the combination of generative text and voice technologies holds promise for further innovation in this space.

In conclusion, the new language model stack is shaping the future of AI applications across industries. Companies are leveraging LLMs to enhance their products, streamline workflows, and improve customer interactions. To make the most of LLMs, here are three actionable pieces of advice:

1. Consider the specific needs of your business: Determine whether a generalized LLM or a customized model is more suitable for your use case. Customization allows for tailored interactions with your data, providing a unique advantage in the market.

2. Keep an eye on emerging tools and frameworks: Stay updated on the latest tools, frameworks, and platforms that facilitate LLM integration. Tools like LangChain can simplify the process of orchestrating and developing LLM applications.

3. Explore the potential of retrieval mechanisms: Consider incorporating retrieval mechanisms, such as vector databases, to enhance the context and quality of LLM results. This approach can help overcome limitations in the model's context window and ensure up-to-date information.

By leveraging the new language model stack and following these recommendations, businesses can unlock the full potential of LLMs and drive innovation in their respective industries.