AutoGen: Enabling next-generation large language model applications

One straightforward way of using built-in agents from AutoGen is to invoke automated chat between an assistant agent and a user proxy agent. As an example (Figure 3), one can easily build an enhanced version of ChatGPT + Code Interpreter + plugins, with a customizable degree of automation, usable in a custom environment and embeddable in a bigger system. It is also easy to extend their behavior to support diverse application scenarios, such as adding personalization and adaptability based on past interactions (e.g., automated continual learning(opens in new tab), teach agents new skills(opens in new tab)).

The agent conversation-centric design has numerous benefits, including that it:

Figure 1. AutoGen enables complex LLM-based workflows using multi-agent conversations.

(Left) AutoGen agents are customizable and can be based on LLMs, tools, humans, and even a combination of them

(Left) AutoGen agents are customizable and can be based on LLMs, tools, humans, and even a combination of them.

(Top-right) Agents can converse to solve tasks. (Bottom-right) The framework supports many additional complex conversation patterns.

With AutoGen, building a complex multi-agent conversation system boils down to:

Defining a set of agents with specialized capabilities and roles.
Defining the interaction behavior between agents, i.e., what to reply when an agent receives messages from another agent.
Both steps are intuitive and modular, making these agents reusable and composable. For example, to build a system for code-based question answering, one can design the agents and their interactions as in Figure 2. Such a system is shown to reduce the number of manual interactions needed from 3x to 10x in applications like supply-chain optimization(opens in new tab). Using AutoGen leads to more than a 4x reduction in coding effort.

Using AutoGen leads to more than a 4x reduction in coding effort.

Capable, conversable, and customizable agents – integrating LLMs, humans, and tools

AutoGen agents have capabilities enabled by LLMs, humans, tools, or a mix of those elements. For example:

One can easily configure the usage and roles of LLMs in an agent (automated complex task solving by group chat) with advanced inference features (e.g., optimize performance with inference parameter tuning).

Human intelligence and oversight can be achieved through a proxy agent with different involvement levels and patterns (e.g., automated task solving with GPT-4 + multiple human users(opens in new tab)).

The agents have native support for LLM-driven code/function execution (e.g., automated task solving with code generation, execution and debugging(opens in new tab), use provided tools as functions(opens in new tab)).