How to Build a Skill-Centric Agent Harness

TL;DR

Build a skill-centric agent harness with a skill registry, a system prompt, and a basic file-reading tool. Treat skill names and descriptions as routing signals, load instructions through progressive disclosure, organize skills around user intent, and rerun evaluations whenever models change. As the library grows beyond a small collection, add search, hierarchy, metadata filters, ownership, versioning, deprecation practices, and human-guided governance.

Transcript

[music] Hi everyone, I am Yogi. I work at Faxet as principal AI engineer. We are a financial data and research company. I'm going to talk about how to build skill centric agentic products and I'm going to post slides so you don't have to keep uh taking photos. So that's my exandle yogi not the bear. Um so let's connect there and uh let's begin. So ... Read More

Key Insights

• Skills are standardized ways to teach AI agents how to perform specific tasks well, while still allowing the underlying model to operate without them. A skill can be a single Markdown file or a more complex package containing referenced files and executable scripts.
• The skill.md file is the heart of a skill because its front matter supplies the name and description used for discovery, while its body contains business logic, operating instructions, and references to supporting files or scripts needed to complete the task.
• A minimal skill-enabled harness requires three components: a skill registry, a system prompt, and a basic file-reading tool. Skills that execute scripts additionally require Bash access or another sandboxed environment capable of running code.
• Progressive disclosure works by placing only each skill's name, description, and path in the system prompt. The agent selects a relevant skill from that compact registry and reads its full instructions only when needed, reducing unnecessary context loading.
• Skill descriptions are routing signals that should align with user requests and contain distinct triggers. A PDF-specific description can route a PDF request correctly, while overlapping, stale, or skill-centered descriptions can cause the wrong skill to run or prevent activation.
• Skill boundaries should follow user intent rather than the underlying data model. Broad workflows such as earnings preparation or a pre-market briefing better reflect real requests than narrowly separated skills for estimates, fundamentals, news, or analyst ratings.
• Skills are model-versioned contracts rather than static documentation because a model upgrade can change which instructions receive attention. Evaluations must be rerun after model changes, even when no skill text has changed, to verify that expected behavior remains intact.
• Skill governance covers admission, ownership, boundaries, life cycle, and coherence. Automated registry gates with human review, named maintainers, semantic versions, deprecation warnings, change logs, audits, hierarchy, and metadata help large libraries remain useful without making governance a bottleneck.

Questions & Answers

Q: What is an agent skill in a skill-centric product?

An agent skill is a standardized way to teach an AI agent how to perform a specific task well. The model can still operate without the skill, but the skill improves how it handles that task. Its core is usually a skill.md file, which can stand alone or reference additional files and executable scripts for more complex workflows.

Q: How do you add basic skill support to an agent harness?

Basic skill support requires a skill registry, a system prompt, and a file-reading tool. The registry records each available skill's name, description, and path. The prompt exposes that compact registry to the agent, and the file tool lets the agent open selected instructions. If a skill executes scripts, the harness also needs Bash or a code-running sandbox.

Q: How does progressive disclosure work for agent skills?

Progressive disclosure gives the agent only the name, description, and path of each available skill at first. The agent uses that information to identify which capability matches the request, then reads the chosen skill's full instructions. This prevents every skill body from being placed in the initial prompt and allows the agent to load only relevant guidance.

Q: Why are skill descriptions important routing signals?

Skill descriptions determine when an agent considers a skill relevant to a user's request. They should use language aligned with user intent and include distinct triggers, such as PDF for a PDF report skill. Descriptions that overlap, focus on internal implementation, or become stale can confuse selection, activate the wrong skill, or leave the appropriate capability undiscovered.

Q: How should an enterprise divide its agent skills?

An enterprise should divide skills around real user intentions rather than its internal data model. For example, users may ask for earnings preparation or a pre-market briefing instead of separate estimates, fundamentals, news, and analyst-rating tasks. Narrow skills can be useful initially, but the library should be refactored as real requests reveal more natural workflow boundaries.

Q: Why do agent skills need evaluations after a model change?

Agent skills need renewed evaluations because changing the model can alter how instructions are interpreted even when the skill itself has not changed. Miraje describes a model that emphasized the beginning of a skill and overlooked critical instructions near its end. Treating skills as contracts tied to a model version makes regression testing essential during upgrades.

Q: How should skill discovery change as a library grows?

A small collection can be listed directly in the system prompt, but that approach becomes less effective as the library expands. With more than about ten skills, the harness can shortlist relevant entries through embedding-based similarity search or a smaller routing model. With hundreds, it needs skill hierarchies, metadata filters, and governance to preserve searchability and coherence.

Q: What governance does a large agent skill library require?

A large library needs governance across admission, ownership, boundaries, life cycle, and coherence. Teams should decide whether a proposed capability deserves a new skill or belongs in an existing one, assign named maintainers, use automated registry gates with human review, apply semantic versioning, issue deprecation warnings, maintain change logs, and periodically audit the collection.

Summary & Key Takeaways

• Agentic products shift features from screens, buttons, forms, and dashboards into skills that guide how an agent completes tasks. Prompts establish who the agent is, tools determine what it can access, and skills encode how work should be performed. Engineers therefore increasingly build reliable harnesses in which product experts can contribute business logic.

• A minimal skill-capable harness needs a registry containing each skill's name, description, and path, plus a system prompt and file-reading capability. The agent first sees compact routing information, then reads only the relevant skill instructions. This progressive disclosure approach avoids loading every skill body while still supporting references, scripts, and complex workflows.

• Enterprise skill libraries require evaluation and governance because model changes can alter instruction-following even when skill files remain unchanged. Small libraries can be listed directly in prompts, while larger collections need retrieval, hierarchy, metadata, admission controls, maintainers, semantic versioning, deprecation warnings, change logs, periodic audits, and human oversight to remain searchable and coherent.