Gemini 3 Review: How Google's New Model Beats GPT-5 and Claude

TL;DR

Gemini 3 outperforms rival frontier models on most major benchmarks, scoring 37.5% on Humanity's Last Exam versus 13% for Claude Sonnet 4.5 and 26.5% for GPT-5.1. Google built it into Search AI mode, a new agent app, and Antigravity, its VS Code fork. It uniquely understands video frame by frame at 1 million input tokens.

Transcript

Gemini 3 is here and my goodness, it was worth the wait. It is an incredible model. And not only that, Google basically built it into every product and service they offer. Google launched Gemini 3, Gemini 3 Pro Preview, and Gemini 3 Deep Think. And let's start with the benchmarks because they absolutely blow the other Frontier models out of the wat... Read More

Key Insights

• Gemini 3 leads on Humanity's Last Exam with 37.5% (no tools) and 45.8% with code execution and search, compared to Claude Sonnet 4.5 at 13% and GPT-5.1 at 26.5%. The older Gemini 2.5 Pro scored 21%.
• Gemini 3 Deep Think spends more tokens in the thinking phase to perform better, scoring 41% on Humanity's Last Exam and 45.1% on ARC-AGI 2, a roughly 10x improvement over Gemini 2.5 Pro on visual reasoning.
• SWE-bench Verified was the only benchmark Gemini 3 did not win, with Claude Sonnet 4.5 taking first at 77.2% and Gemini 3 Pro closely following at 76.2%.
• Gemini 3 uniquely understands video frame by frame rather than just reading the transcript, accepting up to 1 million input tokens and loading full YouTube videos directly via a pasted link.
• Google Search AI mode now dynamically generates custom user interfaces and search result pages written by Gemini 3 based on the user's query.
• Gemini 3 excels at long-horizon planning, reaching over $5,478.16 net worth on Vending Bench 2 versus Claude Sonnet 4.5 in second at around $3,800, staying profitable over a simulated year.
• Antigravity is Google's new agentic coding platform, a VS Code fork competing with Cursor and Windsurf that supports non-Google models including OpenAI's GPT-OSS and Anthropic's Sonnet family.
• Gemini 3 is a brand-new foundation model, not a fine-tune, built as a sparse mixture-of-experts on Google's custom TPU architecture for both pre-training and inference, with a 1 million token context and 64,000 output tokens.

Questions & Answers

Q: How does Gemini 3 compare to GPT-5.1 and Claude Sonnet 4.5 on benchmarks?

Gemini 3 outperforms both on most major benchmarks. On Humanity's Last Exam it scored 37.5% with no tools, versus 13% for Claude Sonnet 4.5 and 26.5% for GPT-5.1. On ARC-AGI 2 it reached 31% against 4 and 17 respectively for competitor models. The only benchmark it lost was SWE-bench Verified, where Claude Sonnet 4.5 took first at 77.2% and Gemini 3 Pro followed at 76.2%.

Q: What is Gemini 3 Deep Think and how well does it perform?

Gemini 3 Deep Think is a variant that spends more tokens in the thinking phase, which lets it perform better. It scored 41% on Humanity's Last Exam versus 37.5% for Gemini 3 Pro, 93.8% on the GPQA scientific knowledge benchmark for first place, and 45.1% on ARC-AGI 2 visual reasoning puzzles, compared to GPT-5.1 at 17% and Claude Sonnet 4.5 at 13.6%, a roughly 10x improvement over Gemini 2.5 Pro.

Q: How does Gemini 3 handle video understanding?

Gemini 3 understands video frame by frame rather than just reading a transcript, which the presenter says is unique among models he has seen. It accepts up to 1 million input tokens and can load a full YouTube video directly from a pasted link because Google owns both YouTube and Gemini. In a demo, it described the exact frame at 3 minutes, identifying a split-screen layout, the presenter's appearance, and legible on-screen text from an Anthropic paper.

Q: What is Antigravity and which models does it support?

Antigravity is Google's new agentic coding platform, a fork of VS Code that competes with Cursor, Windsurf, Replit, and Factory. It is Google's own vibe coding platform and supports not only Gemini models but basically any model you want, including OpenAI's open-source GPT-OSS and Anthropic's Sonnet family. The presenter planned to test it with Gemini 3 Pro High and said a video was coming soon.

Q: How did Gemini 3 perform on the Vending Bench 2 benchmark?

Vending Bench 2 tests a model's ability to control a vending machine, planning inventory, understanding what customers buy, and knowing when to refill, to test AI in real-world economic environments over long time horizons. Gemini 3 finished first with a net worth over $5,478.16, ahead of Claude Sonnet 4.5 in second at around $3,800. Over a simulated year Gemini 3 kept climbing while Gemini 2.5 Pro plateaued and began losing money.

Q: What changes did Google make to Google Search with Gemini 3?

On day zero of the launch, Gemini 3 became available in AI mode within Google Search. It can dynamically generate user interfaces based on a query, producing custom search result pages written by Gemini 3 that previously did not exist. In a demo, turning on AI mode's thinking mode and dropping in a paper generated a dynamic UI. The presenter expects dynamically generated search pages to reach the majority of Google users sooner than expected.

Q: What can the Gemini Agent do in the Gemini app?

The newly updated Gemini app includes Gemini 3 and a Gemini Agent capability that completes real tasks on your behalf and creates UIs dynamically. In a demo, selecting the agent tool and asking it to organize the inbox gave it access to Gmail, where it built a plan, retrieved unread emails, and produced a dynamic view to review and act on emails in bulk. It suggested actions you can accept all, reject, or approve individually, and can write contextual email responses.

Q: What are the technical specifications of Gemini 3 from its model card?

According to Google's model card, Gemini 3 is a brand-new foundation model, not a modification or fine-tune of a prior model. It accepts text, images, audio, and video files as inputs, with a context window of up to 1 million tokens and output of 64,000 tokens. It is a sparse mixture-of-experts model built on Google's own custom TPU architecture for both pre-training and inference, which the presenter calls a huge moat for Google.

Summary & Key Takeaways

• Gemini 3 launched in three forms: Gemini 3, Gemini 3 Pro Preview, and Gemini 3 Deep Think. On benchmarks it leads competitors, scoring 37.5% on Humanity's Last Exam and 31% on ARC-AGI 2 versus far lower scores for Claude Sonnet 4.5 and the GPT-5 family, and 100% on AIME 2025 with code execution.

• Box.com ran its own benchmark on extracting insights from documents and solving multi-step reasoning problems, showing a 22-point jump from Gemini 2.5 Pro (63%) to Gemini 3 Pro (85%), with large gains in healthcare (45 to 94), media (47 to 92), and financial services (51 to 60).

• Beyond the model, Google embedded Gemini 3 across products: Search AI mode with dynamically generated UIs, the Gemini Agent app that can organize your Gmail inbox and draft contextual replies, and Antigravity, a VS Code fork for agentic coding. Gemini 3 is a new mixture-of-experts foundation model on custom TPUs.