What Do Anthropic’s Project Vend, Computer Science Education, and AI Prompts in Papers Reveal About AI?

TL;DR

Anthropic’s Project Vend suggests that an unconstrained AI agent cannot yet reliably run even a small office business. Claudius managed a fridge shop with search, email, and Slack but fell from about $1,000 to about $700 while making inventory, pricing, and Venmo-account errors. The panel argues for hybrid agents built with tools, programmatic checks, bespoke workflows, and human choices about automation. Read on for the specific failures and proposed safeguards.

Transcript

I don't want people to be equating AI and computer science. Computer science as much more than AI. And I will always fall back on saying that the most important thing you could teach people is the basics. Next is critical thinking. All that and more on today's Mixture of Experts. I'm Tim Hwang, and welcome to Mixture of Experts. Each week, MoE brin... Read More

Key Insights

• Project Vend is an Anthropic experiment in which a Claude variant called Claudius operated a small office fridge business using search, email, and Slack. Its responsibilities included maintaining inventory, setting prices, and avoiding bankruptcy over a period of several weeks.
• Claudius was not a successful business operator in the experiment. It began with about $1,000 and finished with about $700, while also showing poor inventory management and occasionally offering products at irrational prices.
• Payment reliability was one of Claudius’s operational weaknesses. The agent asked customers to pay through Venmo, but it eventually hallucinated the account they should use, illustrating how generated information can directly disrupt an otherwise routine business process.
• Scaffolding is a key requirement for practical business agents. The panel described supporting structures such as dedicated inventory tools, programmatic checks, bespoke workflow code, and constraints that keep a language model’s decisions within acceptable operational boundaries.
• Open-ended model control is more ambitious and risky than a carefully designed agent workflow. Allowing a language model to choose all logic and actions can produce creative failures, while a specialized shopkeeping agent could operate within clearly defined tasks, parameters, and risk tolerances.
• Hybrid control is the panel’s preferred direction for agents. A language model can propose plans and creative alternatives, while programmed rules and guardrail flows determine whether those plans violate constraints and whether they should actually be executed.
• Successful demonstrations do not establish general business competence. The panel noted that agent examples often repeat constrained tasks such as ordering airline tickets, and making one familiar workflow function does not mean an agent can reliably handle every operational setting.
• Human agency is a separate consideration from technical capability. The discussion argued that society should ask whether a task ought to be automated, even when automation becomes possible, because workers may want to retain responsibility for activities such as inventory management or procurement analysis.

Questions & Answers

Q: What happened in Anthropic’s Project Vend experiment?

Anthropic placed a Claude variant called Claudius in charge of a small office fridge business for a number of weeks. With access to search, email, and Slack, it maintained inventory, set prices, and tried to avoid bankruptcy. It started with about $1,000 and ended with about $700.

Q: Can AI agents run businesses entirely without guardrails?

Project Vend indicates that relying heavily on a language model for business logic is not yet dependable. Claudius lost money and made routine mistakes involving inventory, prices, and payment information. The panel argued that the model should operate within supporting structures and checks.

Q: Why did Claudius lose money while running the office shop?

Claudius managed inventory poorly and occasionally offered products at irrational prices. It also began hallucinating the Venmo account customers were supposed to use. The discussion did not assign a specific portion of the loss to each error.

Q: What payment error did the Project Vend agent make?

Claudius asked customers to pay through Venmo. After a while, however, it started hallucinating the account they should use for payment. This was one of several routine operational failures observed during the experiment.

Q: What scaffolding did the panel recommend for AI business agents?

The panel recommended structures around the language model, including appropriate inventory-management tools and programmatic checks. It also described agents as potentially using bespoke code to manage workflows around another language model. The model was presented as a key component, but not the whole system.

Q: What did the panel predict about AI agents running businesses by 2027?

Kush Varshney predicted that agents could run businesses by 2027. Gabe Goodhart expected at least one working proof point alongside many examples that would not work well enough for production. Marina Danilevsky predicted that agents would create ways of damaging businesses that humans had not previously considered.

Q: Why is Project Vend evidence of limited rather than general business competence?

The experiment tested whether an agent could operate a very small, rudimentary office business. Claudius was unable to run it successfully, despite having access to search, email, and Slack. Its failure showed that access to general-purpose tools did not supply reliable inventory, pricing, or payment judgment.

Q: What should computer science education emphasize in the age of AI?

The episode argues that AI should not be treated as equivalent to computer science because computer science encompasses more than AI. It identifies the basics as the most important material to teach, followed by critical thinking. That framing preserves foundational education while AI becomes more prominent.

Summary & Key Takeaways

• Anthropic tested an agent called Claudius by placing it in charge of a small office fridge business. With access to search, email, and Slack, it managed inventory and prices while trying to avoid bankruptcy. It nevertheless lost money, ending with roughly $700 after beginning with about $1,000 during the experiment.

• Claudius demonstrated routine operational failures, including weak inventory management, irrational product prices, and a hallucinated Venmo payment account. The panel interpreted these mistakes as evidence that language models should not independently control every business decision. Reliable agents need supporting tools, programmed checks, defined parameters, and workflows designed for their specific operational tasks.

• The discussion favored hybrid agents that combine a language model’s latitude and creative suggestions with controlled flows and enforceable boundaries. The panel also questioned whether every automatable task should be automated. Human agency may remain desirable in areas such as inventory management and procurement, even if technical systems eventually become capable of handling them.