How Does GPT-6 Astra Compare With Rival Models?

TL;DR

GPT-6 Astra reportedly outperforms rival models across scientific, industrial, mathematical, visual, and agentic tasks while often using fewer tokens and lowering costs. Its reduced hallucination rate and efficient action planning strengthen its practical value, but silent reasoning and weaker chain-of-thought monitorability have reportedly unsettled people inside OpenAI and raised concerns for AI safety researchers.

Transcript

GPT6 is here at last and yes, it's yet another AI model. But GPT6 Astra deserves the new number and the new name. And if you know people who haven't checked in on AI in a few weeks or months, their opinions on models may be far more stale than they realize. Because as the title of this video says, Astra is actually a very impressive model, not just... Read More

Key Insights

• GPT-6 Astra reportedly beats Claude Fable 5.1 on difficult benchmarks selected by Anthropic, including Terminal Bench Science and Automation Bench. The transcript also says Astra can achieve these results at lower cost because it uses fewer tokens, making token efficiency an important part of the comparison.
• Terminal Bench Science measures performance on complex scientific investigations with messy data and hidden evaluations. Example tasks include finding repeating brightness dips across thousands of stellar observations, analyzing almost three gigabytes of Greenland lake imagery, and locating and measuring injuries within 237 MRI images.
• Agents Last Exam contains thousands of expert-curated tasks with verifiable outcomes across 55 industries. Its examples require models to operate real industrial tools, plan machining cuts without collisions, work with molten-plastic inputs, and recreate game content that is evaluated by a separate vision model.
• Screen Spot Pro tests whether models can accurately navigate complicated graphical interfaces. Astra reportedly achieves about 92 percent accuracy on software environments that include Adobe Premiere, Photoshop, and Office 365, supporting its ability to perform detailed computer interactions and produce sophisticated visual projects.
• GPT-6 Astra reportedly reaches a peak score of 98 percent on Frontier Math Tier 4, a benchmark where earlier models had scored around zero or much lower. Without an explicit reasoning scratch pad or chain of thought, Astra still reportedly scores 83 percent.
• Astra reportedly contributed a new mathematical starting point for studying distances between prime numbers and improved upon a result associated with Terrence Tao and others by an order of magnitude. The account compares the conceptual novelty to discovering a new chess opening after decades of established practice.
• Astra reportedly reduces hallucinations by roughly three, five, or ten times in scenarios where previous models had hallucinated during real user interactions. The transcript stresses that confabulation has not disappeared, but the measured reduction suggests a meaningful improvement in reliability while creative performance remains strong.
• ARC-AGI 3 reportedly shows Astra solving nearly all tested levels while using fewer actions than the successful-human baseline 96 percent of the time. On average, it uses around 50 percent fewer actions, suggesting that its advantage involves efficient online reasoning rather than repeated random attempts alone.

Questions & Answers

Q: How does GPT-6 Astra compare with Claude Fable 5.1?

GPT-6 Astra reportedly outperforms Claude Fable 5.1 across multiple demanding benchmarks, including tests that Anthropic itself selected to demonstrate Fable's strength. Astra also completes some evaluations with fewer tokens, so its total cost can be lower even when the listed API prices appear similar. The comparison covers scientific work, economically valuable agent tasks, mathematics, and computer interaction.

Q: What kinds of scientific tasks can GPT-6 Astra perform?

GPT-6 Astra is tested on scientific tasks that require writing code, interpreting large or imperfect datasets, and producing answers that can be checked objectively. Examples include finding a small repeating dip in 25,000 stellar brightness readings, explaining lake drainage from almost three gigabytes of Greenland imagery, and identifying, measuring, and labeling an injury across 237 MRI images.

Q: What does Agents Last Exam measure?

Agents Last Exam measures performance on thousands of expert-curated, economically valuable tasks across 55 industries, with outcomes that can be verified. Examples include using industrial machining software, planning cuts that avoid collisions, handling expert inputs about molten plastic, and rebuilding maps, monsters, and battles in an older game while a separate vision model evaluates the resulting environments.

Q: Why is GPT-6 Astra considered more cost-efficient?

GPT-6 Astra is described as more cost-efficient because it can use fewer tokens to complete difficult tasks. Consequently, two models may have apparently similar API prices while producing different total costs for the same evaluation. The transcript reports this pattern on comparisons with Claude Fable models, including Agents Last Exam and Frontier Math Tier 4, where Astra combines stronger results with lower cost.

Q: How well does GPT-6 Astra perform on Frontier Math Tier 4?

GPT-6 Astra reportedly reaches a peak result of 98 percent on Frontier Math Tier 4. The benchmark was created as the hardest Frontier Math tier, and models initially scored around zero. The transcript says Gemini 2.5 Pro and GPT-5 later achieved results around 10, 12, or 20 percent, while Astra also reaches 83 percent without an explicit reasoning scratch pad.

Q: Does GPT-6 Astra still hallucinate?

GPT-6 Astra still hallucinates and has not eliminated confabulation, so its answers cannot automatically be treated as correct. However, OpenAI reportedly tested it on situations where earlier models had hallucinated with real users and observed a substantial decline. The transcript characterizes the reduction as approximately three, five, or ten times, depending on how the comparison is read.

Q: What does GPT-6 Astra's ARC-AGI 3 result show?

GPT-6 Astra reportedly achieves nearly 100 percent on ARC-AGI 3, which places models in unfamiliar game environments requiring abstract pattern recognition and reasoning during interaction. More importantly, Astra uses fewer actions than the successful-human baseline on 96 percent of levels at its maximum setting. Its average action count is around 50 percent lower, indicating unusually efficient problem solving.

Q: Why does GPT-6 Astra raise AI safety concerns?

GPT-6 Astra raises safety concerns because its strong ability to reason silently may make its internal problem-solving process harder to monitor through visible chains of thought. The description says monitorability is losing its hold and notes that AI safety researchers were unnerved. The concern is therefore not limited to capability growth, but includes whether observers can reliably understand or supervise increasingly capable reasoning.

Summary & Key Takeaways

• GPT-6 Astra is presented as a substantial capability advance rather than a routine model update. It reportedly surpasses Claude Fable 5.1 on several difficult benchmarks while using fewer tokens, which can reduce operating costs even when API prices appear similar. The tested work includes science, industrial software, mathematics, vision, and autonomous computer interaction.

• The benchmark examples are designed to represent difficult, verifiable work. Tasks include detecting repeating dips in stellar brightness, analyzing Greenland lake drainage from image data, measuring injuries in MRI scans, planning collision-free machining operations, recreating game environments, and navigating complex professional software. Incorrect hypotheses, collisions, or inaccurate outputs can receive explicit penalties.

• Astra reportedly combines high benchmark performance with fewer hallucinations, strong visual creation, and unusually efficient action planning. It nearly solves ARC-AGI 3 and uses fewer actions than successful humans on most levels. However, its ability to reason without a visible scratch pad raises concerns about whether increasingly capable systems remain understandable and monitorable.