How to rework harnesses for Opus 5.5 and new models

TL;DR

Opus 5.5 expands capabilities for reading, computer use, and long tasks, making harnesses more versatile and cost effective. The landscape now favors multi model verification and output based evaluation, using Opus as a baseline while mixing in Gemini, Luna and others for specific tasks like video, audio, and verification. Reevaluate your harness strategy to leverage SVG generation and cost reductions.

Transcript

This is Oshimo. Today, with the arrival of Opus 5.5, GPT-6, Sol, and Luna, the landscape of AI models is really shifting. I think we all have a lot to reconsider and reevaluate when it comes to building harnesses, so I’d like to dive into that in this video. Right. As for benchmarks, I’ve been paying close attention to the knowledge markers lately,... Read More

Key Insights

• Opus 5.5 expands capabilities in reading, computer use, and long running tasks.
• The harness landscape is shifting toward multi model verification and output based evaluation.
• Opus 5.5 enables detailed SVG based video creation and improved cost performance.
• Gemini 3.8 Flash and MiMo V2.6 Flash are used for video and audio verification tasks.
• Luna is noted as highly capable for large scale, routine tasks.
• Costs are decreasing across models, making large scale verification more feasible.
• A mix of Opus, Gemini, Luna and Astra provides a balanced, task oriented approach.
• OSWorld and computer use benchmarks show improved practical usability of AI in verification tasks.

Questions & Answers

Q: What changes in the model landscape are highlighted by Opus 5.5 and new models?

Opus 5.5, along with models like GPT-6 Sol and Luna, shifts the balance of power among AI models, expanding what can be handled by harnesses. The speaker notes improvements in reading accuracy, long running task handling, and cost reductions, which together broaden the feasible use cases for AI driven development and verification.

Q: How has the cost profile changed with Opus 5.5 and related models?

The video emphasizes that Opus 5.5 and related models have seen cost reductions, enabling higher throughput and more extensive use without sacrificing quality. This price trend supports more aggressive verification strategies, including output based reviews and multi model checks, and reduces the barrier to scale deployment.

Q: Which models are recommended for video and audio verification tasks?

For video and audio verification, the speaker recommends using Gemini Flash 3.8 and MiMo V2.6 Flash, noting their usefulness in media input tasks. These tools complement Opus by handling modalities that Opus might not cover as efficiently, enabling a broader verification workflow.

Q: What is the role of Luna in the new harness strategy?

Luna is described as highly effective for mass, routine tasks, with strong capability in handling large data volumes. In a mixed harness strategy, Luna provides scalability for repetitive verification and data processing, helping to maintain quality while increasing throughput.

Q: Why is SVG based video creation relevant to harnesses?

SVG based video creation is relevant because Opus 5.5 improves the ability to generate and replicate visuals precisely, enabling detailed, SVG driven outputs. This strengthens the reliability of visual tasks and supports verification workflows that require accurate graphic reproduction.

Q: What should teams consider when mixing models in a harness?

Teams should consider balancing capabilities and costs by using Opus as the baseline while integrating Gemini for media input, Luna for high volume tasks, and Astra or Fable for image oriented work. The aim is to maintain a target quality while expanding capabilities across modalities.

Q: How does the speaker view verification going forward?

Verification is moving toward output based evaluation and screen or image based checks, supported by lower costs and more capable vision features. The talk suggests increasing use of automated reviews by separate agents, leveraging tool chains like Playwright or Agent Browser for scalable verification workflows.

Q: What practical steps can teams take immediately based on this video?

Immediately, teams can re-evaluate their harness design to incorporate Opus 5.5 as a core, plan for multi model verification, and experiment with Gemini Flash and MiMo for video and audio tasks. They should also budget for Luna in high volume scenarios and progressively mix in Astra or Fable as needed for images, while leveraging cost reductions to scale testing and verification.

Summary & Key Takeaways

• Harness design should adapt to Opus 5.5 improvements in readability, computer use, and long running tasks to enable broader automation. The new landscape supports mixed model usage for verification and output based evaluation, leveraging tools like Playwright or Agent Browser for efficient reviews.

• Cost reductions and higher usability in Opus 5.5 open opportunities for high volume tasks and multi model verification. A multi model approach becomes more practical as models gain capabilities in video, audio and visual verification while maintaining affordability.

• A practical harness now combines Opus as the baseline with selective use of Gemini Flash for media input, Luna for mass tasks, and Astra or Fable for image tasks to balance cost and performance.