How to Build AI Products From Model Capabilities

TL;DR

Build AI products through bottom-up experimentation close to the model, because important capabilities often become clear late in development. Start with real user problems, prototype before imposing abstractions, and evaluate useful outcomes instead of benchmarks alone. As coding agents accelerate implementation, product teams must also remove organizational delays and develop stronger approaches to review, oversight, and architecture.

Transcript

You all might know Mike of course as chief product officer of Anthropic, but you were also Sequoia founder at one point. Is that right? Yeah. And for a hot week. For a hot week. And what was that company? It was Instagram. Instagram. Thank you. Welcome everyone. Mike Lauren, take it from here. Thank you for joining, Mike. Yeah, happy to be here. He... Read More

Key Insights

• AI product development is most effective when experimentation happens close to the model, because teams may discover important capabilities late in the process. Krieger contrasts this approach with Instagram's more top-down planning cycles, which typically operated across a three-to-six-month timeframe.
• Real user problems remain the foundation of valuable AI products. Developer tools should enable people to do something useful, novel, and fast, while end-user products should address the needs people actually have rather than relying on model capability alone.
• Artifacts began as a research prototype before a designer and an engineer transformed it into a production product. The example shows how Anthropic nurtures ideas that originate through bottom-up discovery instead of requiring every significant product to begin with an executive plan.
• MCP originated from repeated integration work involving Google Drive, GitHub, and a planned third implementation. Anthropic recognized that these projects shared the task of bringing context into models, then created an abstraction after encountering the same underlying problem multiple times.
• Open protocols gain practical value through participation beyond their original creator. Anthropic made MCP genuinely open because it saw value in standardization, while collaboration with larger organizations introduced authentication, identity management, and Exchange-related requirements that were not initially treated as priorities.
• Agentic systems are moving from retrieving context toward taking actions and automating workflows. Krieger identifies action-oriented integrations and communication among agents as promising areas, while cautioning that patterns for agent-to-agent interaction are still too early to standardize extensively.
• AI coding quality depends on outcomes and maintainability, not benchmark performance alone. Krieger says Anthropic examines whether generated code is code that people like working with, while also considering how generation fits into larger codebases and teams rather than only informal vibe coding.
• Faster code generation magnifies organizational inefficiencies because meetings and alignment delays block far more potential implementation work than before. Anthropic is consequently confronting questions about review, architectural dead ends, technical debt, and the appropriate point for human oversight.

Questions & Answers

Q: How should teams build products around emerging AI models?

Teams should begin with genuine user problems, then allow researchers, designers, and engineers to experiment close to the model. Model capabilities may become apparent relatively late, so rigid top-down plans can miss valuable discoveries. Krieger recommends a more bottom-up creative process in which promising prototypes can develop before the organization determines how to refine, productionize, and support them.

Q: Why does bottom-up development suit AI products?

Bottom-up development suits AI because researchers and builders often discover what a model can do during experimentation rather than knowing every useful capability in advance. Krieger says this required him to unlearn the more top-down, three-to-six-month planning approach used at Instagram. Anthropic's Artifacts product illustrates the alternative: a research prototype was taken forward by a designer and an engineer and shipped to production.

Q: How did Anthropic develop MCP?

MCP grew from Anthropic's experience building separate Google Drive and GitHub integrations, followed by plans for another bespoke implementation. The team recognized that the integrations had a common purpose: bringing context into a model. Following Krieger's pattern of doing something three times before identifying an abstraction, two engineers prototyped a shared protocol, improved it, and made it open for broader adoption.

Q: Why did Anthropic make MCP an open protocol?

Anthropic believed MCP would create more value as a broadly standardized protocol than as something owned only by Anthropic. Opening it allowed a wider community to iterate on the design and contribute requirements. Work with companies such as Microsoft and Amazon surfaced complex concerns involving authentication, identity management, and Exchange servers, which were not among Anthropic's initial priorities but mattered for broader practical adoption.

Q: What is the next stage for MCP and AI integrations?

The next stage is moving beyond context retrieval toward systems that can take actions and automate workflows. Krieger points to integrations that can pull information from GitHub and launch Zapier actions as early examples. He also expects interaction among agents to become important, including the possibility that agents could hire other agents, although he considers current agent-to-agent patterns too early for extensive standardization.

Q: How should teams evaluate AI-generated code?

Teams should evaluate whether AI-generated code produces good outcomes and remains code that people want to work with, rather than relying only on model benchmarks. Krieger distinguishes limited vibe coding from the demands of maintaining an entire codebase with a large team. Effective evaluation must therefore include usability, review quality, architectural direction, and the role generated code plays in collaborative development.

Q: How extensively does Anthropic use AI-generated code?

Krieger says more than half of Anthropic's pull requests are generated with Claude Code and estimates that the share may already exceed 70 percent. This heavy internal use makes Anthropic an early test case for the benefits and problems of coding agents. It also forces the company to examine code review, oversight, architectural risks, and the consequences of having AI review AI-generated pull requests.

Q: Why do coding agents expose product organization problems?

Coding agents increase implementation speed, which makes existing organizational delays more costly. Krieger explains that an alignment meeting may no longer obstruct only one hour of engineering work, but the equivalent of four or eight hours. As code generation accelerates, product organizations must reconsider coordination, review, and decision-making processes so those processes do not become the primary constraints on delivery.

Summary & Key Takeaways

• Mike Krieger argues that familiar product fundamentals still apply to AI: teams must solve real problems and meet users where they are. What changes is the planning process. Because model capabilities can emerge late, Anthropic allows researchers, designers, and engineers to develop prototypes from the bottom up before establishing broader product direction.

• MCP emerged after Anthropic built separate Google Drive and GitHub integrations and recognized that both performed the common task of bringing context into a model. Engineers developed an abstraction after repeated implementations, opened the protocol for adoption beyond Anthropic, and expanded it through feedback from organizations with complex authentication and identity requirements.

• AI coding should be judged by whether it produces code people want to work with and achieves good outcomes, not only by benchmark scores. Anthropic uses Claude-generated code in most pull requests, which raises unresolved questions about code review, architectural oversight, technical debt, and whether faster implementation can compensate for traditional organizational inefficiencies.