How Do OpenAI o3 and o4-mini Use Tools?

TL;DR

OpenAI o3 and o4-mini use tools iteratively, combining visual inspection, web browsing, Python, and terminal access to complete complex, multistep tasks. In the featured physics demonstration, o3 examined a 2015 poster, reconstructed a missing result near 1.2, and compared it with recent literature. Read on for the models’ benchmark results, differences, and practical tool-use workflow.

Transcript

OpenAI just dropped two brand new models, 03 and 04 mini. These are incredible cuttingedge models, but the best part and what seems to be maybe in my opinion a new scaling law is that they have access to full tool usage. And we not only got 03 and 04 mini, we got the one more thing surprise from Open AI and they dropped a brand new project which lo... Read More

Key Insights

• Tool access is the central capability highlighted for o3 and o4-mini. Unlike earlier reasoning-model releases that lacked tools at launch, these models can use browsing, Python, terminals, and visual inspection while working through complex tasks iteratively rather than relying on a single isolated tool call.
• Iterative tool use allows o3 to adjust its approach as new information appears. In the physics demonstration, it inspected a poster, found the relevant plot, inferred missing calculations, searched recent research, and compared its reconstructed result with current literature in one extended workflow.
• o3 is presented as the strongest of the two released models, while o4-mini is described as a faster and cheaper alternative. The transcript notes that the naming may be confusing because the full o4 model was not part of the release being discussed.
• Competition mathematics performance is reported at 91% for o3 and 93% for o4-mini on AIME 2024 without tools. On AIME 2025, o3 reached 88% and o4-mini reached 92%, compared with 79% for o1 and 86% for o3-mini.
• Coding performance is one of the clearest areas of improvement. Both o3 and o4-mini reportedly achieved a Codeforces score of 2700 with terminal access, while SWE-bench Verified results reached 69% for o3 and 68% for o4-mini, compared with 48% for o1.
• Tools improve performance on Humanity’s Last Exam. The transcript reports o3 rising from 20% without tools to 25% with Python and browsing, while o4-mini rises from 14% to 17%, supporting the argument that agentic scaffolding expands what reasoning models can accomplish.
• Multimodal reasoning is stronger in the reported evaluations. o3 scored 87% on MathVista and 75% on chart reasoning, while o4-mini scored 84% and 72%, respectively. The chart-reasoning result for o3 represents a sizable increase from the reported 55% score for o1.
• The presenter’s model-development theory is speculative rather than confirmed. He proposes that OpenAI may be using different checkpoints from a progressively trained base model, then applying reinforcement learning with verifiable rewards to produce the reasoning behavior observed across o3, o3-mini, o4, and o4-mini.

Questions & Answers

Q: How do OpenAI o3 and o4-mini use tools?

They can repeatedly select and use tools during a single reasoning process instead of making one tool call and immediately returning an answer. The transcript describes visual inspection, web browsing, Python, and terminal use, allowing the models to gather evidence, perform calculations, and revise their approach as a task develops.

Q: What are the main differences between OpenAI o3 and o4-mini?

The transcript presents o3 as the stronger model overall and o4-mini as a faster, cheaper alternative. Despite its smaller size, o4-mini outperformed o3 on the reported no-tool AIME tests, while o3 led on broader evaluations involving instruction following, visual reasoning, and complex agentic work.

Q: What did o3 accomplish in the physics poster demonstration?

o3 examined a 2015 physics poster about the proton isovector scalar charge and recognized that the requested final result was missing. It found the relevant plot, extrapolated an unnormalized value, applied a normalization constant, and browsed recent literature. Its estimate was close to the researcher’s published result of around 1.2, although it had more uncertainty than recent results.

Q: How well did o3 and o4-mini perform on mathematics benchmarks?

On AIME 2024 without tools, o3 scored 91% and o4-mini scored 93%, compared with 74% for o1 and 87% for o3-mini. On AIME 2025, o3 scored 88% and o4-mini scored 92%, while o1 scored 79% and o3-mini scored 86%.

Q: How strong are o3 and o4-mini at coding tasks?

With terminal access, both models reportedly achieved a Codeforces score of 2700. On SWE-bench Verified, o3 scored 69% and o4-mini scored 68%, compared with 48% for o1 and 49% for o3-mini. The reported code-editing evaluations also showed substantial gains.

Q: How much did tools improve performance on Humanity’s Last Exam?

o3 improved from 20% without tools to 25% with Python and browsing. o4-mini rose from 14% to 17% with those tools. These results support the transcript’s argument that tool access expands what reasoning models can accomplish.

Q: What visual reasoning results were reported for o3 and o4-mini?

On MathVista, o3 scored 87% and o4-mini scored 84%. On chart reasoning, o3 reached 75% and o4-mini reached 72%, while o1 scored 55%. The physics demonstration also showed o3 zooming into and exploring a scientific poster to locate relevant evidence.

Q: How might OpenAI have developed the o3 and o4 model family?

The presenter speculates that OpenAI may use different checkpoints from a progressively trained base model and then apply reinforcement learning with verifiable rewards. Under this theory, increasingly capable checkpoints could underpin o3, o3-mini, o4, and o4-mini. The transcript does not present this as a confirmed explanation from OpenAI.

Summary & Key Takeaways

• OpenAI introduced o3 and o4-mini as reasoning models with tool access available from launch. Their defining capability is iterative tool use: they can inspect images, browse the web, run calculations, and use terminals across multiple steps. The transcript argues that this agentic behavior matters alongside improvements in raw model intelligence.

• A scientific demonstration showed o3 examining a physics poster, recognizing that the requested final result was absent, estimating a value from a plotted slope, applying a normalization constant, and browsing recent literature for comparison. The model produced an estimate close to the researcher’s published result while identifying its greater uncertainty relative to newer work.

• Benchmark results presented in the transcript show improvements across competition mathematics, programming, PhD-level science, visual reasoning, software engineering, code editing, and instruction following. The presenter attributes much of the value to tool use and agentic scaffolding, while speculating that different model versions may come from progressively trained base-model checkpoints enhanced through reinforcement learning.