What is MCP and how can Anthropic fix it?

TL;DR

MCP has large context problems that bloat prompts and slow down agents. Anthropic is proposing dynamic tool loading and on demand tool definitions to reduce context waste and improve agent usefulness. The video argues current MCP implementations are flawed and highlights new advanced tool use approaches as a potential path forward.

Transcript

I try my best to not talk too much about buzzwordy, annoying things that I don't see much value in. And that's why I only have one video about model context protocol or MCP as many of y'all know it. I just don't see that much value in the standard yet. And I do see a lot of problems that it causes. That's why I did a video about how much I think it... Read More

Key Insights

• MCP requires loading all tool context into the model, which bloats prompts and wastes tokens.
• The key to scalable AI agents is to discover, learn, and execute tools dynamically rather than preloading all tools.
• On demand tools can reduce cost and improve speed by keeping only relevant definitions in context for a given task.
• Dynamic tool loading aims to separate tool usage from long term context to avoid token bloat.
• Tool definitions when included in each step cause repeated token usage across turns, increasing cost.
• SDK style tool integration allows code to call tools, which can be more compact than natural language tool descriptions.
• Agents should load tools as needed during execution to minimize unnecessary context without sacrificing capability.
• Anthropic is exploring advances that could decouple tool availability from upfront context loading, potentially improving efficiency.
• More videos from this channel:
  • How to avoid fetch on render in React data fetching
  • Why Do AI Costs Keep Rising Across Models?
  • What is GPT6.1 Soul and why it costs less
• More from this channel: Theo - t3․gg

Questions & Answers

Q: What is MCP and why is it problematic in practice?

MCP refers to a context protocol that forces models to carry large tool definitions and context in every interaction. This results in bloated prompts, higher token usage, slower responses, and increased costs. The speaker argues that such upfront context makes agents less efficient and harder to scale as tool libraries grow, creating a fundamental bottleneck in real world use.

Q: What is Anthropic attempting to improve with advanced tool use?

Anthropic is introducing advanced tool use on the Claude developer platform with features that let models discover, learn, and execute tools dynamically. This approach attempts to avoid loading all tools into context upfront, instead allowing the model to fetch and utilize tools as needed during the task, which should reduce token waste and improve efficiency.

Q: Why does the presenter criticize loading full MCP context in every prompt?

The presenter explains that including every tool definition and related context in each prompt mimics reading a dictionary from the start, forcing the model to process a lot of irrelevant information. This bloats the context window, increases token usage, and slows down the loop of tool calls, ultimately making the system more expensive and less effective.

Q: How might dynamic tool loading improve agent performance?

Dynamic tool loading would let agents load only the tools relevant to the current task, keeping the context lean. This reduces token usage per turn, lowers latency, and keeps the model focused on the task rather than processing unnecessary tool definitions. It aligns with the idea of tools being loaded on demand rather than preloaded.

Q: What is the problem with always including tool definitions in every interaction?

Including tool definitions in every interaction leads to repeated, unnecessary context being sent back and forth. Each turn may incur multiple tool calls, and the full tool definitions are re-sent, wasting tokens and increasing cost. This also makes caching and efficient reasoning harder for the model.

Q: What alternative to natural language tool calls is discussed?

An alternative discussed is an SDK style approach where tools are provided as code rather than long textual descriptions. This can allow the agent to call tools more efficiently and reduce the surface area of context that must be parsed and transmitted, potentially improving performance and reducing token usage.

Q: What analogy does the speaker use to describe MCP context waste?

The speaker compares MCP context to a dictionary where you must read every entry from the first word on every lookup, which is inefficient and off topic for the current search. This illustrates how unnecessary context slows progress and distracts from the actual task.

Q: What is the envisioned end state for AI agents according to the video?

The end state envisions AI agents that can seamlessly work across thousands of tools without having every tool definition loaded upfront. Agents would discover and load tools on demand, and use code execution with SDK like tool definitions to keep context minimal while maximizing tool capability and task success.

Summary & Key Takeaways

• MCP context handling creates inefficiencies by forcing all tool definitions into every prompt, making interactions slow and costly.

• Anthropic proposes on demand tool discovery and dynamic loading to keep context lean and relevant.

• The speaker critiques MCP current practice and advocates for tool driven, SDK based approaches to improve agent performance.