How Does Kimi K2 Thinking Outperform GPT-5?

TL;DR

Kimi K2 Thinking outperforms GPT-5 on Humanity’s Last Exam and agentic browsing by combining long, coherent reasoning with extensive tool use. The open-source, open-weights model scored 44.9 versus GPT-5’s 41.7 on Humanity’s Last Exam and can execute up to 200–300 sequential tool calls without human interference. Read on for benchmark comparisons, limitations, and examples of its mathematics and coding capabilities.

Transcript

We may have another deepseek moment on our hands. Moonshot Labs, a Chinese frontier AI company, just released a completely open-source, completely open weights, frontier level model that is better than GPT5, better than Claude 4.5 on some of the hardest benchmarks. Let me break it all down for you. And this video is brought to you by Vulture. More ... Read More

Key Insights

• Kimmy K2 is an open-source AI model developed by Moonshot Labs, outperforming GPT-5 and Claude 4.5 on challenging benchmarks.
• The model is capable of executing 200-300 sequential tool calls, demonstrating advanced reasoning abilities.
• Kimmy K2 achieved a score of 44.9 on humanity's last exam, surpassing GPT-5's score of 41.7.
• In agentic browsing and search tasks, Kimmy K2 scored 60.2, outperforming GPT-5 and Claude 4.5.
• The model's training cost was approximately $5.6 million, reflecting the decreasing costs of developing frontier AI models.
• Kimmy K2 has a unique style in creative writing and can handle complex logical problems with ease.
• The model is highly efficient, with 32 billion parameters active during inference, despite having a trillion total parameters.
• China's open-source AI models are rapidly closing the gap with frontier models, showcasing significant advancements in AI technology.

Questions & Answers

Q: How does Kimi K2 Thinking outperform GPT-5?

Kimi K2 Thinking scored 44.9 on Humanity’s Last Exam, compared with GPT-5 at 41.7. It also scored 60.2 on the BrowseComp agentic search benchmark versus GPT-5 at 54.9, although GPT-5 performed better on SWE-bench Verified and LiveCodeBench v6.

Q: What is Kimi K2 Thinking?

Kimi K2 Thinking is a completely open-source, open-weights thinking model released by Moonshot Labs, a Chinese frontier AI company. It is designed to reason step by step for extended periods while using tools during its reasoning process.

Q: How many sequential tool calls can Kimi K2 Thinking execute?

The model can execute up to 200–300 sequential tool calls without human interference. It can also reason coherently across hundreds of steps while working on complex problems.

Q: How does Kimi K2 Thinking perform on Humanity’s Last Exam?

Kimi K2 Thinking achieved a score of 44.9 on Humanity’s Last Exam. The transcript compares that with 41.7 for GPT-5 and 32 for Claude Sonnet 4.5 Thinking.

Q: How well does Kimi K2 Thinking perform in agentic browsing and search?

Kimi K2 Thinking scored 60.2 on the BrowseComp agentic search benchmark. GPT-5 scored 54.9 and Claude Sonnet 4.5 Thinking scored 24.1 in the comparison.

Q: Does Kimi K2 Thinking outperform GPT-5 and Claude Sonnet 4.5 on every benchmark?

No. On SWE-bench Verified, Kimi K2 scored 71, behind GPT-5 at 74 and Claude at 77. On LiveCodeBench v6, it scored 83.1, below GPT-5 at 87 but above Claude Sonnet 4.5 Thinking at 64.

Q: Can Kimi K2 Thinking solve advanced mathematics problems with tools?

The demonstration shows it solving a PhD-level mathematics problem with 23 tool calls in its reasoning chain. It repeatedly searched the web for references, reasoned with the retrieved information, and ultimately produced the correct answer.

Q: What coding projects did Kimi K2 Thinking create from prompts?

The examples include a component-heavy word-processing clone with working formatting and local document saving. The model also produced a gradient-descent explainer visualization and a bloodstream simulation with adjustable virus and white-blood-cell settings, with the visualization described as coming from a single prompt.

Summary & Key Takeaways

• Kimmy K2, developed by Moonshot Labs, is an open-source AI model that outperforms GPT-5 and Claude 4.5 on several benchmarks. It excels in executing complex reasoning tasks and agentic browsing, demonstrating its advanced capabilities in AI reasoning and tool usage. The model's training cost was around $5.6 million, highlighting the decreasing cost of developing frontier AI models.

• Kimmy K2 achieved a score of 44.9 on humanity's last exam, surpassing GPT-5's score of 41.7. It is capable of executing 200-300 sequential tool calls, showcasing its advanced reasoning abilities. The model also scored 60.2 in agentic browsing and search tasks, outperforming GPT-5 and Claude 4.5.

• China's open-source AI models, like Kimmy K2, are rapidly closing the gap with frontier models, demonstrating significant advancements in AI technology. Kimmy K2 has a unique style in creative writing and can handle complex logical problems with ease, making it a highly efficient and advanced AI model.