Is Kimi K2 Thinking the Best Tool-Calling Model?

TL;DR

Kimi K2 Thinking stands out most for tool calling, where it can reportedly complete 200–300 consecutive calls without human intervention. Its benchmark scores, low usage cost, and English writing are impressive, but the model is extremely large, slow, token hungry, and inconsistent on coding implementation details, while its open-weight status comes with licensing caveats.

Transcript

been a bit since we got a new openweight model that was worth looking at. And boy is what moonshot cooking worth looking at. This is the best tool calling model ever from what I have seen. Yes, not just openweight model period. It's crushing on humanity's last exam getting a state-of-the-art score there as well as on browser comp. And it's neck and... Read More

Key Insights

• Kimi K2 Thinking is a trillion-parameter open-weight model whose published weights occupy approximately 594 gigabytes in an INT4-quantized form. Its enormous size limits practical local deployment, and Moonshot was the only observed provider hosting it through OpenRouter at the time discussed.
• Tool calling is Kimi K2 Thinking's clearest strength, with the model reportedly capable of making 200–300 consecutive tool calls without human intervention. The presenter considers it the strongest tool-calling model he has observed, including both open-weight and proprietary alternatives.
• Kimi K2 Thinking is the leading open-weight model in the cited Artificial Analysis ranking and the second-highest-scoring model overall on its intelligence index. The presenter cautions that this index is imperfect, but still treats the model's placement as meaningful evidence of strong general capability.
• Reasoning efficiency is a major weakness because Kimi consumed 140 million tokens during the Artificial Analysis intelligence index evaluation, including 130 million reasoning tokens. GPT-5 on high reasoning used 82 million reasoning tokens, while Sonnet 4.5 used only 34 million in comparison.
• Kimi K2 Thinking's benchmark cost was about $380 despite its unusually high token consumption. That made it comparable in price to Claude 4.5 Haiku in the discussed evaluation, although the presenter found Kimi less dependable for coding work than its impressive overall benchmark position might suggest.
• The standard Moonshot endpoint is inexpensive but slow, costing $0.60 per million input tokens and $2.50 per million output tokens at roughly 18 tokens per second. The turbo endpoint reaches about 85 tokens per second but costs $1.15 for input and $8 for output.
• Coding execution is inconsistent even when Kimi creates apparently relevant project files. In the presented Next.js test, it modified the layout and CSS and created components, but failed to mount those components on the homepage, leaving the default starter page visible until explicitly corrected.
• English writing quality is a notable strength because Kimi produced a structured, persuasive defense of Java without falling into the extensive bullet-list format seen in the presented GPT-5 and Sonnet comparisons. Its response directly contrasted outdated complaints about Java with features of modern Java.

Questions & Answers

Q: What is Kimi K2 Thinking best at?

Kimi K2 Thinking appears strongest at tool calling and long autonomous workflows. The model can reportedly complete 200–300 consecutive tool calls without requiring human intervention, which the presenter describes as better than any other model he has observed. It also performs strongly on Humanity's Last Exam, BrowserComp, and coding benchmarks, although its practical coding reliability is less consistent.

Q: How large is Kimi K2 Thinking?

Kimi K2 Thinking is a trillion-parameter model, and its open weights occupy approximately 594 gigabytes. The available weights are INT4 quantized, which should make execution easier than using an unquantized version, but the trillion-parameter scale still creates substantial hosting challenges. At the time discussed, Moonshot was the only provider the presenter had seen running it through OpenRouter.

Q: How much does Kimi K2 Thinking cost to use?

Moonshot's standard Kimi K2 Thinking endpoint costs $0.60 per million input tokens and $2.50 per million output tokens, with performance around 18 tokens per second and sometimes lower. The turbo endpoint costs $1.15 per million input tokens and $8 per million output tokens, but increases throughput to approximately 85 tokens per second.

Q: Why is Kimi K2 Thinking considered token hungry?

Kimi K2 Thinking used 140 million total tokens during the cited Artificial Analysis intelligence index evaluation, the highest usage reported there. Approximately 130 million were reasoning tokens. For comparison within the transcript, GPT-5 on high reasoning used 82 million reasoning tokens, while Sonnet 4.5 used 34 million, showing that Kimi spends substantially more tokens reaching its answers.

Q: Is Kimi K2 Thinking good for coding?

Kimi K2 Thinking scores competitively with GPT-5 and Sonnet 4.5 on evaluations such as SWE-bench Verified and LiveCodeBench, but practical implementation can be unreliable. In one Next.js test, it created components and changed layout and CSS files but never rendered the components on the homepage. The presenter had to remind it to complete that essential integration step.

Q: How fast is Kimi K2 Thinking?

The standard hosted version is slow, operating at roughly 18 tokens per second in the cited OpenRouter listing, with the presenter observing even lower performance at times. A turbo version reaches approximately 85 tokens per second but costs significantly more. One coding request reportedly required around ten minutes, and Moonshot attributed earlier performance problems to IP bandwidth rather than GPU availability.

Q: How does Kimi K2 Thinking compare with GPT-5 and Sonnet 4.5?

Kimi K2 Thinking is described as neck and neck with GPT-5 and Sonnet 4.5 on several coding benchmarks, while appearing especially strong at tool calling. It uses far more reasoning tokens than either comparison model in the cited evaluation. In practical coding it missed an important integration step, but its English writing response was more compelling and less list-heavy in the demonstrated test.

Q: What are the main drawbacks of Kimi K2 Thinking?

Kimi K2 Thinking's main drawbacks are its enormous 594-gigabyte weight package, slow standard endpoint, heavy reasoning-token consumption, and uneven coding follow-through. Faster inference is available only through a substantially more expensive turbo endpoint. The presenter also flags licensing as an important catch associated with its open-weight release, although the provided transcript does not include the detailed licensing explanation.

Summary & Key Takeaways

• Kimi K2 Thinking is a trillion-parameter, 594-gigabyte open-weight reasoning model from Moonshot. It achieves leading open-weight benchmark results and competes closely with GPT-5 and Sonnet 4.5 on several coding evaluations. Its most distinctive capability is sustained autonomous tool use, reportedly handling 200–300 consecutive calls without human assistance.

• The model has serious efficiency limitations. Moonshot's standard service costs $0.60 per million input tokens and $2.50 per million output tokens while delivering about 18 tokens per second. Its turbo service reaches about 85 tokens per second but raises prices to $1.15 for input and $8 for output.

• Practical testing produced mixed results. Kimi generated multiple files for a Next.js interface but failed to render its components on the homepage until reminded, exposing weak implementation follow-through. Its English prose was more compelling and less dependent on bullet lists than comparable GPT-5 and Sonnet responses in the presented Java writing test.