The New Language Model Stack: Building AI-powered Applications

The New Language Model Stack: Building AI-powered Applications

Almost every company in the Sequoia network is embracing the use of language models in their products. These models have proven to be incredibly versatile and have been integrated into various applications across different industries. From code auto-complete features to chatbots and workflow optimization, language models are revolutionizing the way we interact with technology.

The new stack for these applications revolves around language model APIs, retrieval mechanisms, and orchestration. Open source usage is also on the rise, with more companies experimenting and implementing these models into their products. In fact, 65% of companies already have language model applications in production, showcasing the growing adoption of this technology.

One of the key components of the new stack is the use of retrieval mechanisms, such as vector databases. These mechanisms help provide relevant context to the language model, resulting in higher quality results and reduced inaccuracies. Companies are using purpose-built vector databases or leveraging existing options like pgvector or AWS offerings to implement this functionality.

Another important aspect of the stack is the use of an LLM orchestration and application development framework like LangChain. This framework allows companies to prototype and deploy language model applications effectively. Adoption of such frameworks has increased in recent months, indicating their value in the development process.

Monitoring LLM outputs, cost, and performance, as well as conducting A/B testing of prompts, are areas that currently have lower interest among companies. However, as more large companies and regulated industries adopt language models, we can expect an increase in interest in these areas. Additionally, there is growing interest in exploring complementary generative technologies that combine generative text and voice.

While the majority of companies rely on foundation model APIs like OpenAI's GPT, there is a significant portion (15%) that is building custom language models from scratch or using open source options. This trend highlights the desire for companies to customize language models to their unique contexts and data. Custom model training has also seen meaningful growth, indicating the need for specialized compute, model hubs, and training frameworks.

In conclusion, the new language model stack is transforming the way companies develop and deploy AI-powered applications. Language model APIs, retrieval mechanisms, and development frameworks are the pillars of this stack, with open source usage and custom model training on the rise. Companies are leveraging these technologies to create personalized and context-aware experiences for their users. As the field continues to evolve, it's essential for companies to stay up-to-date with the latest advancements and explore ways to tailor language models to their specific needs.

Actionable Advice:

1. Evaluate the suitability of language models for your product or industry: Assess how language models can enhance your existing products or enable new use cases in your industry. Identify areas where language models can provide value and explore potential integration options.

2. Consider the use of retrieval mechanisms for improved results: Implement retrieval mechanisms, such as vector databases, to provide relevant context to your language models. This can help improve the quality of results, reduce inaccuracies, and address data freshness issues.

3. Explore custom model training for personalized experiences: If your use case requires highly customized language models, consider building your own models from scratch or leveraging open source options. Invest in the necessary infrastructure, training frameworks, and experiment tracking tools to effectively train and tune your models.

Software Architecture for Building Serverless Microservices

Serverless architecture is gaining popularity for its scalability, cost-effectiveness, and ease of use. When building serverless microservices, it's essential to consider the software architecture that supports the development and deployment of these applications. This article explores the key components and considerations for building serverless microservices.

Event-driven architecture plays a crucial role in serverless development, enabling the decoupling of components and facilitating asynchronous communication between services. Event producers, such as web applications, microservices, and IoT devices, create and publish events, while event consumers subscribe to events and handle them in near-real time.

Event carriers, also known as event routers or event buses, accept events from producers and deliver them to subscribers. They provide support for event filtering, transformation, routing, and successful delivery to target consumers. Event carriers play a vital role in managing the flow of events in a distributed system.

Serverless architecture promotes the use of microservices, which are independently deployable services modeled around a specific business domain. These microservices can be composed and integrated using APIs, enabling modular and scalable systems. API-first thinking is crucial in serverless development, where interfaces play a significant role in designing and integrating services.

Several software architecture patterns can be applied when building serverless microservices. Hexagonal architecture, Onion architecture, Clean Architecture, and MACH architecture are some of the popular patterns that promote loose coupling, separation of concerns, and modular system design.

Event sourcing is another important pattern in serverless development, allowing you to reconstruct the state of an application based on a stream of events. It involves capturing and storing events in an event store, which can be managed by a dedicated microservice. Event sourcing is particularly useful for auditing, tracing, and maintaining the integrity of critical business flows.

When building serverless microservices, it's important to consider the structure and categorization of events. Events can be categorized into domain events, operational events, AWS events, internal events, local events, transformed events, and custom events. Each event type serves a specific purpose and contributes to the overall functionality of the system.

EventBridge, a fully-managed serverless event bus, provides several features and capabilities for event-driven architectures. It supports the creation of custom event channels, event schema registry, event scheduling, and event pipes for one-to-one integration pipelines. EventBridge simplifies event management and enables efficient communication between services.

In conclusion, building serverless microservices requires careful consideration of the software architecture that supports event-driven development, modular system design, and effective event management. By leveraging event-driven architecture, API-first thinking, and appropriate software patterns, you can create scalable, decoupled, and efficient serverless applications.

Actionable Advice:

1. Embrace event-driven architecture: Design your microservices to be event-driven, allowing for asynchronous communication and decoupling of components. Identify event producers and consumers within your system and leverage event carriers to manage event flow.

2. Choose appropriate software patterns: Explore software architecture patterns like Hexagonal architecture, Onion architecture, Clean Architecture, and MACH architecture to design modular, scalable, and maintainable serverless microservices. Consider the specific needs and constraints of your application when selecting a pattern.

3. Consider event sourcing for critical business flows: If your application requires auditing, tracing, and maintaining the integrity of critical business flows, consider implementing event sourcing. Capture and store events in an event store, and use a dedicated microservice to manage event ingestion, translation, and data retention policies.

By combining the power of language models and serverless architecture, companies can build AI-powered applications that are scalable, customizable, and efficient. The new language model stack, with its focus on language model APIs, retrieval mechanisms, and development frameworks, provides a solid foundation for developing context-aware and personalized experiences. Meanwhile, serverless microservices offer a flexible and cost-effective approach to building scalable and modular applications. By leveraging event-driven architecture, appropriate software patterns, and effective event management, companies can unlock the full potential of serverless microservices and language models to drive innovation and enhance user experiences.