How to orchestrate AI agents in software projects

TL;DR

AI is shifting software teams from manual coding to designing orchestration systems. The main ideas are spec driven development, context engineering, loop engineering, and using sub agents to investigate problems. Security and understanding how LLMs work are essential for safe, reliable AI workflows that scale with projects.

Transcript

Just recently, I was talking to a software engineer and made conversation. He stops to check his phone, approve a code change and continue talking. It turns out he had multiple agents running in the background for all of the different roles that you would typically have on a team. So he had a developer, a code reviewer, an operations and many, many... Read More

Key Insights

• AI will shift the role of developers from hand coding to architectural orchestration and governance of AI agents.
• Spec driven development replaces ad hoc prompts with clear requirements that AI agents can fulfill, improving consistency and traceability in project delivery.
• Context management is essential to optimize AI performance, accounting for model context windows and external data tools used during development.
• Loop engineering replaces back-and-forth prompting with recursive goals where the AI iterates until a task is complete.
• Sub agents enable parallel exploration of problems, such as library behavior, and report results to a central agent to accelerate problem solving.
• Understanding how LLMs work, including GPU memory and KV cache, gives developers a practical advantage in building efficient AI workflows.
• Prompt injections pose real security risks; disposable sandboxes reduce exposure and protect credentials during AI-assisted coding tasks.
• The future view, echoed by AI pioneer references, is that AI acts like electricity, expanding the engineering workforce to maintain, scale, and secure AI-enabled systems.

Questions & Answers

Q: How does spec driven development change AI coding projects

Spec driven development changes AI coding projects by moving from prompting an agent to execute ad hoc tasks to sending a complete specification of requirements and desired outcomes. This approach enables AI agents to build the project artifacts, define milestones, and work toward measurable goals without constant human prompting. It improves reliability and reduces back and forth.

Q: What is the shift from implementing to conducting in software teams

The shift from implementing to conducting means developers focus on high level goals, architecture, and system design rather than writing functions by hand. AI models can perform the actual coding tasks, so developers supervise, orchestrate, and steer progress, ensuring alignment with long term objectives and architectural constraints while milestones are completed by agents.

Q: How does context engineering help AI in development

Context engineering is about providing the AI with reusable, contextual information about the codebase and project. It includes skills, automation, and external data integrations that accompany each AI request. This makes agents more efficient, reduces redundant queries, and improves accuracy by giving the model relevant background during problem solving.

Q: What role do loop engineering and sub agents play

Loop engineering creates a system where goals are defined and the AI iterates autonomously until completion, reducing human back and forth. Sub agents extend capability by investigating specific components or libraries and returning results to the main agent. Together, they enable scalable task decomposition and faster problem resolution.

Q: Why is it important to understand how LLMs work for AI coding

Understanding LLMs is important because it explains limitations in time perception and memory, how weights and KV caches work, and how models are stored and retrieved during inference. This knowledge helps developers design better prompts, manage expectations, optimize performance, and build more efficient, reliable AI driven software.

Q: How can teams mitigate prompt injection risks in AI coding

Teams mitigate prompt injection risks by running coding agents inside disposable sandboxes, isolating the agent from sensitive credentials and external networks. This containment reduces attack surface, protects secret data, and makes it safer for AI agents to access necessary information while preserving system integrity and user trust.

Q: What did Andrew Ng mean by AI being the new electricity

The reference to AI as the new electricity suggests AI will power many industries and routines, just as electricity transformed daily life. In software, this implies more engineers focused on maintenance, scale, reliability, and governance of AI systems, while automation handles routine coding, enabling creative and architectural work.

Q: What is the expected future of software engineering with AI agents

The future of software engineering with AI agents involves more orchestration, governance, and architecture rather than manual coding. Engineers will design goals, risk management, and system interactions for AI agents to implement. Those who excel in critical thinking and system design will be most valuable as capabilities scale and complexity grows.

Summary & Key Takeaways

• A system of AI agents changes how software is built by shifting focus from writing code to coordinating agents that handle tasks across the project lifecycle, from requirements to deployment.

• Developers become conductors or captains, prioritizing architectural decisions and milestones that AI agents can execute, guided by clear specifications and efficient context management.

• Security, sandboxing, and understanding model limitations are critical as teams adopt AI agents, ensuring reliability, integrity, and safe interactions with sensitive data and external systems.