Is Gemini 3.8 Flash the Best Low-Cost AI Model?

TL;DR

Gemini 3.8 Flash offers strong software engineering and specialized legal performance at a low introductory price of $0.75 per million input tokens and $3.75 per million output tokens. Its results vary sharply by task, so organizations should compare its cost per completed task and test it against internal workloads instead of choosing it solely from headline benchmark rankings.

Transcript

Google is back. They just dropped Gemini 3.8 Flash and it's actually good. And that's not the most impressive thing about it. So, I'm going to show you the benchmarks in a second, but I just want to give you a little bit of context as to where Google is currently in the competitive landscape. So, a few years ago, Chat GPT came out and Google was no... Read More

Key Insights

• Gemini 3.8 Flash scored 73.7% on Deep SWE V1.1, placing it effectively even with Claude Opus 5 and slightly above GPT 5.6 Soul at 72.7% on the long-horizon software engineering benchmark highlighted in the evaluation.
• Gemini 3.8 Flash scored 1545 on GDPval, below Claude Opus 5 at 1824 and GPT 5.6 Soul at 1710. The result suggests that its performance on real-world knowledge work, including document extraction, analysis, and presentations, is solid but not category-leading.
• Gemini 3.8 Flash has introductory prices of $0.75 per million input tokens and $3.75 per million output tokens. The transcript states that these prices expire at the end of the year, after which the listed rates are $1.50 and $7.50.
• The Harvey legal benchmark placed Gemini 3.8 Flash first at 61.4%, with Gemini 3.7 Flash in second place. Its combination of high legal-agent performance and low pricing makes it a particularly relevant candidate for law firms to evaluate on their own work.
• Terminal coding performance varies substantially by benchmark version. Gemini 3.8 Flash led Terminal Bench 2.1 at 89.4%, but scored only 19.1% on the much harder Terminal Bench 4.0, trailing Claude Opus 5 and GPT 5.6 Soul while beating Sonnet 5.
• Gemini 3.8 Flash ranked first on Humanity's Last Exam with a score of 55.9, yet reached only 59% on OSWorld computer-use tasks. Claude Opus 5 scored 75% on OSWorld, illustrating how model rankings can change significantly across capabilities.
• Cost per task is more informative than token price alone because it includes both published token rates and the number of tokens consumed to solve a task. A model priced at half the rate can cost the same overall if it needs twice as many tokens.
• Gemini 3.8 Flash Cyber scored 86.2 on CyberGym, compared with 85.6 for GPT 5.5 Cyber and 83% for both GPT 5.6 Soul and Mythos 5. Access is restricted to trusted defenders through the Fair Wind program, according to the transcript.
• More videos from this channel:
  • What Is Jev and How Does Its AI Model Work?
  • How to visualize room designs with AI in minutes
  • How to speed up AI workloads with ultrafast GPT and open models
• More from this channel: Matthew Berman

Questions & Answers

Q: How much does Gemini 3.8 Flash cost?

Gemini 3.8 Flash has an introductory price of $0.75 per million input tokens and $3.75 per million output tokens. The transcript says these introductory rates expire at the end of the year, with stated later prices of $1.50 for input and $7.50 for output. Even at those later rates, the presenter describes it as cheaper than the comparable GPT 5.6 Terra model.

Q: How well does Gemini 3.8 Flash perform on coding benchmarks?

Gemini 3.8 Flash delivers mixed but sometimes leading coding results. It scored 73.7% on Deep SWE V1.1, slightly above GPT 5.6 Soul at 72.7% and effectively even with Claude Opus 5. It also led Terminal Bench 2.1 at 89.4%, but achieved only 19.1% on the more difficult Terminal Bench 4.0, below Claude Opus 5 and GPT 5.6 Soul.

Q: Is Gemini 3.8 Flash good for legal work?

Gemini 3.8 Flash appears especially promising for legal work based on the reported Harvey legal benchmark. It ranked first with 61.4%, while Gemini 3.7 Flash placed second. The presenter argues that law firms should examine it because it combines strong legal-agent performance with low token prices. However, firms should still test it against internal legal tasks before making an adoption decision.

Q: Why should companies create internal AI benchmarks?

Companies should create internal benchmarks because Gemini 3.8 Flash performs exceptionally well on some tests and only moderately on others. Public rankings may not represent a company's documents, workflows, quality standards, or recurring tasks. Testing competing models on representative internal work allows an organization to select the right model for each job instead of automatically adopting the latest release from one provider.

Q: What is cost per task for an AI model?

Cost per task combines a model's price per million input and output tokens with the number of tokens it actually uses to complete a task. This is more useful than comparing published rates alone. If one model costs half as much per token but consumes twice as many tokens for the same result, its effective task cost is approximately the same.

Q: How did Gemini 3.8 Flash perform on knowledge and computer tasks?

Gemini 3.8 Flash scored 1545 on GDPval, a benchmark covering real-world knowledge work such as PDF extraction, data analysis, and presentation creation. That trailed Claude Opus 5 at 1824 and GPT 5.6 Soul at 1710. On OSWorld computer and browser control, Gemini scored 59%, which the presenter considered respectable, while Claude Opus 5 led with 75%.

Q: What is Gemini 3.8 Flash Cyber?

Gemini 3.8 Flash Cyber is a specialized model built for cyber capabilities and offered only to trusted defenders through the Fair Wind program. The transcript describes it as having fewer cyber guardrails than the normal Gemini 3.8 Flash model. It scored 86.2 on CyberGym, exceeding GPT 5.5 Cyber at 85.6 and the reported 83% scores for GPT 5.6 Soul and Mythos 5.

Q: How good were Gemini 3.8 Flash's generated web pages and 3D scenes?

The practical demonstrations were mixed. Its seven low-poly biomes looked good overall, particularly the beach and farm, but included glitches and less detail than GPT 5.6 Soul. The DGX Spark webpage looked strong and had interactive values, while the Apple page was overly simple and lacked useful product selection. The rubber duck design also compared poorly with more childlike alternatives.

Summary & Key Takeaways

• Gemini 3.8 Flash combines low pricing with uneven but sometimes leading benchmark performance. It scored 73.7% on Deep SWE V1.1, 61.4% on the Harvey legal benchmark, and 55.9 on Humanity's Last Exam, while delivering less competitive results on GDPval, Terminal Bench 4.0, and OSWorld.

• The introductory API pricing is $0.75 per million input tokens and $3.75 per million output tokens. The stated later pricing is $1.50 and $7.50. Cost per task remains the more meaningful measure because an inexpensive model can lose its advantage if it consumes substantially more tokens to finish identical work.

• Practical demonstrations produced mixed results. Gemini created respectable low-poly biomes and a strong DGX Spark webpage, but its Apple and rubber duck designs were comparatively weak. These examples reinforce the broader benchmark pattern: Gemini 3.8 Flash can excel at selected tasks without being the strongest general-purpose option for every workload.