Why Does Claude’s Advisor Strategy Pair Opus With Sonnet or Haiku?

TL;DR

Claude’s advisor strategy uses Sonnet or Haiku as the executor and calls Opus only when a task needs stronger reasoning. In evaluations, Sonnet with Opus improved SWE-bench performance by 2.7 percentage points while reducing cost per agentic task by almost 12%. Read on to compare model prices, understand the Messages API limitation, and decide when this setup makes sense.

Transcript

So, Enthropic just gave us the adviser strategy. This lets you pair Opus as an adviser with a cheaper model like Sonnet or Haiku as the executor. And what this does is it allows us to have near Opus level intelligence in your agents at a fraction of the cost because it's not a matter of which model is best. Obviously, Opus 4.6 is the most capable. ... Read More

Key Insights

• The advisor strategy pairs Opus with cheaper models like Sonnet or Haiku.
• Opus is used only for complex tasks, reducing overall costs.
• Sonnet with Opus as an advisor improved performance by 2.7% on SWE bench.
• Using Haiku with Opus more than doubled the performance on browse comp.
• The messages API allows defining tools and sending queries to Claude.
• Cloud Code is a finished product using the same models as the API.
• Opus plan mode in Claude Code helps optimize session usage.
• Testing various prompts helps decide the best model setup for cost efficiency.

Questions & Answers

Q: How does Claude’s advisor strategy work?

The strategy assigns Sonnet or Haiku as the executor and lets it call Opus when advisor-level intelligence is needed. This avoids using Opus for every step when a cheaper model can complete the easier work just as effectively.

Q: How much did the advisor strategy improve Sonnet’s performance and cost?

In the evaluations described, Sonnet with Opus as its advisor gained 2.7 percentage points on SWE-bench compared with Sonnet alone. It also reduced the cost per agentic task by almost 12%.

Q: How did Haiku perform with Opus as an advisor?

Haiku with Opus as its advisor scored 41.2% on BrowseComp, compared with Haiku’s solo score of 19.7%. The setup cost more than Haiku alone but remained cheaper than completing the task with only Opus.

Q: How much do Opus, Sonnet, and Haiku cost per million tokens?

Opus costs $5 per million input tokens and $25 per million output tokens. Sonnet costs $3 for input and $15 for output, while Haiku costs $1 for input and $5 for output.

Q: Is the advisor strategy available directly in Claude Code?

The transcript says the actual advisor strategy function exists only in the Messages API. Claude Code is a finished coding assistant built on top of the API, although Opus plan mode can help reserve Opus for situations where it is needed.

Q: What can the Claude Messages API do?

The Messages API can send questions, receive answers, analyze images, PDFs, and documents, and define tools that Claude can call. It is intended for apps, chatbots, tools, automations, and products that need control over their AI experience.

Q: What can’t the Claude Messages API do by itself?

By itself, the Messages API cannot read files on your computer, edit code, run terminal commands, or browse the web. It is also stateless, so it does not remember information between requests unless developers build that functionality themselves.

Q: When should developers use the advisor strategy?

It is useful when a task mixes difficult and simple steps, so Opus is unnecessary for the entire workflow. For example, Opus can advise on a difficult first step while Haiku handles two easier steps at one-third of the cost described for Opus.

Summary & Key Takeaways

• The advisor strategy involves using Opus as an advisor and cheaper models like Sonnet or Haiku as executors. This setup allows handling complex tasks with Opus while assigning simpler tasks to less costly models, optimizing performance and reducing expenses. It demonstrated significant cost savings and performance improvements in AI model evaluations.

• The messages API and Claude Code both utilize Opus, Sonnet, and Haiku models. The API is suitable for building apps with AI integrations, while Claude Code serves as a coding assistant. Opus plan mode in Claude Code helps manage session limits by using Opus only when necessary, ensuring efficient resource utilization.

• Testing with various prompts helps determine the best model combination for specific tasks, balancing cost savings and quality. The advisor strategy is beneficial for applications requiring AI, offering a cost-effective solution without compromising performance, and can be implemented in projects via GitHub for further experimentation.