How to ship thousands of PRs with AI agents

TL;DR

The video shows how potato built a scalable workflow with AI agents by distilling his own skills into agent skills and using a trust ladder to delegate more tasks. He emphasizes rigorous verification and reducing manual bottlenecks, enabling rapid output such as thousands of PRs per month and high project velocity.

Transcript

We're live. Okay, sweet. I think we're live, everybody. Right. What I'll do is I'm going to cut this down on YouTube a little bit later and then we'll have like a proper I'll give it like a proper intro and I'll, you know, introduce you and all that stuff. So, I'm going to hide you from the main chat and then I'll bring you back in. Does that sound... Read More

Key Insights

• The key to scaling with agents is distilling personal expertise into a set of reusable skills that can be executed by AI agents.
• Trust in agents grows as you implement rigorous verification and reduce unnecessary prompts or tautologies, increasing reliability.
• Distinguishing between the easy thing and the best outcome helps avoid shortcuts and improves overall quality.
• Expressing intent clearly becomes the bottleneck as models grow more capable, making domain knowledge exceptionally valuable.
• Effective agent work relies on strong prompt engineering and careful handling of language to guide thinking traces.
• Reducing tautological tests prevents wasteful validation and speeds up iteration cycles.
• Early experiences with building skills and an agent brain teach the importance of tooling, metrics, and visibility for performance.

Questions & Answers

Q: How did potato begin climbing the trust ladder with AI agents?

Potato began by recognizing that micromanaging a single agent was inefficient and that skills could be distilled into a broader set of reusable actions. He then focused on measuring output and impact, which led to building a more structured approach to agent capabilities. This groundwork allowed him to gradually grant more autonomy to agents as trust increased.

Q: What role did his time at Meta and burnout play in this approach?

His time at Meta exposed him to high pace engineering and complex systems, while burnout pushed him to pursue a side project. This combination motivated him to streamline his workflow, convert personal techniques into agent skills, and seek more scalable, less labor-intensive methods to maintain velocity without sacrificing quality.

Q: Why is domain expertise considered valuable even with advanced AI agents?

Domain expertise remains valuable because as models grow smarter, the bottleneck shifts to articulating intent and goals clearly. A deep, specific understanding of a domain allows you to design precise prompts and workflows that agents can execute reliably, yielding higher quality results with greater velocity.

Q: What is the importance of verification in potato’s workflow?

Verification is crucial because it ensures agents do not shortcut or produce suboptimal results. By implementing rigorous checks and testing strategies, the workflow maintains high quality and predictable outcomes, which is essential when scaling to thousands of PRs and coordinating complex tasks across teams.

Q: How does language and prompt engineering influence agent behavior?

Language shapes how agents think and plan. Prompt engineering helps the agent understand nuances, priorities, and constraints. By carefully choosing words and framing tasks, you guide the agent’s reasoning, reduce ambiguity, and align its actions with intended outcomes.

Q: What is tautology in testing, and why should it be avoided with agents?

Tautology in testing refers to redundant or trivial tests that do not add new information about system behavior. Avoiding tautology in agent workflows helps prevent wasted effort, focuses validation on meaningful outcomes, and accelerates iteration while maintaining confidence in results.

Q: How does the concept of a skills directory help scale AI capabilities?

A skills directory structures and abstracts individual capabilities into reusable modules. This makes it easier to compose complex workflows, share capabilities across teams, and upgrade agents incrementally. It also provides clear metrics for performance and a roadmap for expanding agent capacity.

Q: What practical steps can teams take to start implementing this approach?

Teams should start by documenting and extracting their routine knowledge into discrete skills, build a minimal viable brain for agents, implement rigorous verification, and then gradually increase agent autonomy with monitored control. Prioritize clarity of intent and invest in prompt design to improve agent alignment with goals.

Summary & Key Takeaways

• Potato describes identifying bottlenecks in manual agent management and turning expertise into reusable skills to scale workflows.

• The discussion highlights a trust ladder concept, where greater trust in agents enables more complex tasks and higher throughput.

• The conversation covers language and prompts as core tools for shaping agent behavior and improving outcomes.