How to Build an AI Harness for Repeatable Workflows

TL;DR

An AI harness is code wrapped around an AI agent to make it more effective at a specific job. It combines tailored context, controlled actions, and defined outcomes so repetitive workflows can run consistently with less prompting, while permissions can restrict activities such as editing files or contacting customers unless explicitly approved.

Transcript

A harness is some code around an AI agent that makes it more effective. Why we've seen people build these specific use case harnesses is sometimes with a specific job, you just want to micromanage a little bit. You just want to be more prescriptive about how that job gets done. [music] I'm going to show you how it works and then we will talk about ... Read More

Key Insights

• An AI harness is code surrounding an AI agent that improves its effectiveness for a particular use case. The surrounding code may contain AI, but it does not have to, because its purpose is to structure how the underlying agent performs a job.
• The three core components of a harness are specific context, specific actions, and specific outcomes. Together, these elements tell the agent what information matters, which operations it may perform, and what results or artifacts it must produce before the workflow is complete.
• A harness is most useful when the same workflow repeatedly needs the same setup and outcomes. It can combine deterministic process rules with nondeterministic AI reasoning, making it suitable for complex jobs that follow a recognizable sequence and depend on a defined collection of tools.
• Specialized harnesses reduce repeated prompting by encoding the user's intent in advance. In the demonstration, pasting a Sentry issue into the terminal interface starts the expected investigation process without requiring a fresh explanation of the task or its standard operating rules.
• Permission controls are a practical benefit of a custom harness. An investigation-only mode can prevent source-file changes, while separate flags can authorize editing, changing inputs, or messaging customers only when those actions have been explicitly selected and approved.
• The demonstrated debugging harness uses the Claude Agent SDK as its core and a terminal interface built with Ink. It connects to operational tools including Sentry, Linear, GitHub, and Vercel so the agent can gather evidence and support the required follow-up work.
• Consistent artifacts make harness output useful beyond the immediate agent session. The debugging workflow can document work in Linear, create a specific report, and produce follow-up documentation that other engineering team members can use after an issue has been investigated or fixed.
• AI harnesses can support technical and nontechnical workflows beyond debugging. Examples given include managing production incidents, preparing pull requests for release, handling support escalations, managing migrations, conducting research through a prescribed method, and consolidating documents through a consistent process.

Questions & Answers

Q: What is an AI harness?

An AI harness is code placed around an AI agent to make the agent more effective for a specific use case. The code supplies the context, available actions, process constraints, and expected outcomes needed for a particular job. It may include AI components, but that is optional. Its defining purpose is to make the underlying agent more useful and consistent.

Q: When should you build an AI harness?

You should build an AI harness when a recurring workflow requires the same setup, tools, process, permissions, and outcomes. It is especially relevant when a job combines deterministic steps with AI reasoning and is complex enough to benefit from a prescribed structure. Examples include debugging incidents, preparing releases, handling escalations, managing migrations, structured research, and document consolidation.

Q: What are the three essential parts of an AI harness?

The three essential parts of an AI harness are specific context, specific actions, and specific outcomes. Context gives the agent the information required for the job. Actions define the tools and operations it can use. Outcomes specify what completion should look like, including reports, documentation, fixes, or other artifacts that must result from the workflow.

Q: Why use a custom harness instead of a general-purpose coding agent?

A custom harness can encode intent and workflow rules that would otherwise need to be repeated in prompts to a general-purpose coding agent. It can start from a simple input, such as a Sentry issue link, then follow the expected investigation process automatically. This can improve efficiency and consistency while ensuring that required follow-up actions and artifacts happen every time.

Q: How can an AI harness control agent permissions?

An AI harness can restrict which tools and operations an agent is allowed to use. For example, an investigation-only mode can let the agent explore evidence and explain a root cause without modifying source files. Separate flags can permit source edits, input changes, or customer messages only when the user explicitly selects and approves those more consequential actions.

Q: How does the Sentry debugging harness work?

The Sentry debugging harness accepts a Sentry issue as the task input and starts an investigation run through a terminal interface. It launches a Claude Agent SDK session, gathers evidence, streams activity, and develops a root-cause hypothesis. Depending on the selected permissions, it can remain investigation-only or support additional actions, while also producing reports and follow-up artifacts.

Q: Which tools and technologies are used in the debugging harness?

The harness core runs on the Claude Agent SDK, while its custom terminal interface is built with the Ink library. It connects to Sentry, Linear, GitHub, and Vercel for evidence gathering and task execution. The description also states that GPT-5.5 and Claude Opus were used while building the harness code itself.

Q: What other workflows can an AI harness manage?

An AI harness can manage any sufficiently repetitive and structured workflow that benefits from fixed context, controlled actions, and consistent outcomes. Examples identified include production incident management, preparing pull requests for release, support escalations, migrations, structured research, and document consolidation. The approach is not limited to coding, technical tasks, or terminal-based interfaces.

Summary & Key Takeaways

• An AI harness adds purpose-built code around an agent so it can perform a specific job more effectively. Its essential components are relevant context, permitted actions, and intended outcomes. The approach is most useful when a recurring workflow requires the same setup, process, tools, permissions, and deliverables each time.

• The demonstrated harness investigates Sentry issues for ChatPRD through a custom terminal interface built with Ink. Its core uses the Claude Agent SDK and connects with Sentry, Linear, GitHub, and Vercel. It gathers evidence, develops a root-cause hypothesis, streams activity, and creates artifacts that can support the wider engineering team.

• A specialized harness provides more control than repeatedly prompting a general-purpose coding agent. It can infer intent from a pasted issue link, enforce investigation-only or editing modes, require approval for sensitive actions, and consistently produce reports or Linear documentation. The same pattern can support releases, escalations, migrations, research, and document consolidation.