What Are Code Agents and Why Do They Beat JSON Agents?

TL;DR

An agent is any application where an LLM has some control over the execution flow, forming a continuum of agency from simple routers to multi-step loops. Code agents, which write their actions as executable code rather than JSON tool calls, handle loops, parallelization, and variables far more efficiently than JSON-based agents.

Transcript

hey everyone welcome to another lining pod uh I'm aleso partner in CTO at deible and I'm joined by m swix founder of small AI hey and today it's a very beautiful small episode because we have the creator of small agents Emer from hugging face welcome hi everyone thanks for having me Emer what is an agent so that's that's a really good question like... Read More

Key Insights

• An agent is an application where an LLM has some level of control over the execution flow of the app, according to Harrison Chase's definition, so agency exists on a continuum rather than as a binary yes-or-no property.
• Agency has different layers: an if-else routed by an LLM output is a router, a while loop whose continuation depends on LLM output is a multi-step agent, and executing a function from LLM output is function calling.
• Smolagents was built to be very simple because existing agent libraries were often too complicated to work with; its core agents.py file is under 10,000 lines so the logic stays apparent and readable.
• Code agents write their actions in code, unlike most other agents which use JSON. This combination of simplicity plus acting in code was the driving motivation behind building smolagents over Christmas.
• The CodeAgent class is smolagents' go-to class, while the ToolCallingClass follows the classic approach of plugging into OpenAI tool-calling formats that return a dictionary with a tool name and tool arguments.
• The CodeAct paper shows code agents outperform JSON: a code agent can run a for loop making multiple tool calls in precise sequence, whereas the same workflow in JSON might require around 20 separate actions that are hard to parallelize.
• Code agents make defining a variable trivial by assigning it to a name, whereas doing the same with a JSON agent is extremely difficult, giving code additional advantages beyond parallelization.
• Running LLM-written code has big security challenges, so smolagents reuses a custom Python interpreter built from the ground up and uses E2B for sandboxing, with Docker support also being built.

Questions & Answers

Q: What is an AI agent?

According to Harrison Chase's definition cited in the video, an agent is an application where an LLM has some level of control over the execution flow of your app. Because an LLM's knowledge is limited to its highly compressed training data, you build an agent to give it access to the outside world. The degree of control the LLM has over execution determines how agentic the application is, so it is not a binary property but a continuum.

Q: What are the different layers or levels of agency?

Agency comes in different layers depending on how much control the LLM has over execution flow. If you have an if-else where the branch taken is determined by an LLM output, that is a router with some level of agency. If you have a while loop whose continuation depends on LLM outputs, that is a multi-step agent. If you execute a function based on an LLM output, that is function calling. This forms a continuum rather than a one-zero definition.

Q: Why is smolagents designed to be so simple?

Roucher built smolagents to be simple because whenever he tried to work with an agent library he found it often too complicated to work with, which was a key pain point and the reason for the 'smol' in the name. The main agents.py file at the core of the library is kept under 10,000 lines of code so it is very easily understandable. The goal is also to make the logic apparent so you can read through the code and see how it works easily.

Q: What is the difference between code agents and JSON agents?

Code agents write their actions in executable code, while JSON agents use the classic tool-calling approach where the model returns a dictionary with a tool name key and tool arguments key. The CodeAgent class is smolagents' core go-to class, whereas the ToolCallingClass plugs into OpenAI tool-calling formats. Roucher found the JSON approach highly suboptimal, which is why he designed code agents, based on the opinion that code agents work better.

Q: Why do code agents work better than JSON agents according to the CodeAct paper?

The CodeAct paper shows that with a code agent, an LLM can run a for loop performing multiple different tool calls, precisely controlling how parallel loops run and in what sequence each part executes. Doing the same workflow in JSON actions could require around 20 different actions and would be really hard to parallelize. Code also makes defining a variable easy by assigning it to a name, whereas with a JSON agent that is very difficult.

Q: What is the origin story of smolagents?

Smolagents grew out of transformers.agents, which was its experimental version started earlier by Aymeric's colleagues Lassander, HuggingFace's head of Open Source, and Silvan. They had built a code agent that could perform one step, including a custom Python interpreter built from the ground up to handle the security challenges of code agents. Roucher reused that interpreter, made it multi-step with state persisting between steps, and worked through Christmas because he thought it was a big thing.

Q: How does smolagents handle the security risks of running LLM-generated code?

Running LLM-written code has big security challenges, so smolagents relies on a custom Python interpreter that was built from the ground up as part of the earlier transformers.agents work. For sandboxing, the team currently uses E2B, which they chose because it starts really quickly, seems well supported, and appears to be a robust solution. They are also building Docker support. Their approach is to take the simplest route that gives a solid result and build it quickly.

Q: What is the HuggingFace agents course about?

The agents course is run by another team at HuggingFace, with Roucher supporting it. It grew so popular that at some point it broke the website used for sign-ups, exceeding the roughly 10K subscribers of past courses. The course starts from the basics, teaching what an agent is and how to code one from scratch before covering a few frameworks. This from-scratch approach helps learners avoid getting lost in the specifics of a predefined framework. It also offers a certification people can show on their resume.

Summary & Key Takeaways

• Aymeric Roucher of HuggingFace, creator of smolagents, joins the Latent Space podcast to discuss what agents are, how to build them, and how to evaluate them. He frames an agent using Harrison Chase's definition: an application where an LLM has some control over the app's execution flow.

• Agency is a continuum, not binary. A router uses LLM output in an if-else, a multi-step agent uses it in a while loop, and function calling executes functions from LLM output. Roucher built smolagents to be simple, with a core file under 10,000 lines, because other libraries felt too complicated.

• Smolagents emphasizes code agents that write actions as code rather than JSON, citing the CodeAct paper showing code handles loops, parallelization, and variables better. It grew from transformers.agents, reuses a custom Python interpreter, uses E2B for sandboxing, and supports an upcoming HuggingFace agents course.