How Do AI Agent Context and Skills Work?

TL;DR

AI agents produce better results when they receive relevant context without repeatedly loading unnecessary instructions. Keep always-on agent files for proprietary or workflow-critical information, use skills for specialized procedures through progressive disclosure, and build each skill by completing the workflow with the agent step by step, correcting failures before documenting the proven process.

Transcript

Ross, Mike, welcome back to the pod. By the end of this episode, what are people going to learn? I hope I'm going to share some wisdom on how you can use the agents better. There's a lot of information going on right now. I disagree with most of it and that's what we're going to talk about. So, at the end, whether you're building something, using a... Read More

Key Insights

• Modern AI models are already exceptionally capable, according to the discussion, but their results still depend heavily on context. The same model can produce high-quality work or poor output depending on which instructions, resources, tools, code, and conversation history are assembled for the task.
• An agent's context is assembled from several sources: a provider system prompt, persistent instruction files, available skill descriptions, tool definitions, relevant project files, and the user conversation. Together, these components consume the context window and influence what the model can consider while completing an action.
• Persistent agent instruction files are included on every conversational turn. A thousand-line file might consume about 7,000 tokens repeatedly, so the speaker argues that most users should avoid such files unless they contain proprietary knowledge or a methodology required in every interaction.
• Skills use progressive disclosure to reduce unnecessary context. Only the skill's name and description are initially available to the agent, while its detailed instructions remain unloaded until the agent recognizes that the current request requires that particular skill.
• A skill is better suited than a persistent instruction file for specialized procedures such as generating a particular report or structuring code in a specific way. The agent can load those instructions only when the matching task occurs instead of carrying them throughout every unrelated exchange.
• Effective skills are developed by performing the workflow with the agent before formalizing it. The user should guide individual steps, observe mistakes, answer questions, and add corrections until the agent can reproduce the desired process consistently.
• Vague instructions can cause consistently shallow judgments even when the underlying model is strong. In the sponsor example, asking the agent to research companies produced universal approval until the workflow specified sources to check and conditions that triggered automatic rejection.
• Context management becomes increasingly important as conversations grow. The transcript describes an initial context of roughly 20,000 tokens potentially expanding toward a 250,000-token limit, after which Claude Code and OpenAI Codex compact the conversation to continue operating within their available context.

Questions & Answers

Q: How does context affect the quality of AI agents?

Context determines which instructions and information an AI agent can use while acting. The discussion identifies the provider's system prompt, persistent instruction files, skill descriptions, tool definitions, project files, and conversation history as major components. Even when the underlying model is highly capable, irrelevant or repetitive context can consume available tokens, while focused context can steer the same model toward a much better result.

Q: What information fills an AI agent's context window?

An AI agent's context window can contain a general system prompt from the model provider, persistent files such as agent.md or claude.md, the names and descriptions of available skills, definitions for tools, relevant codebase material, and the ongoing user conversation. These elements accumulate as work continues, so the complete context can grow from an initial collection of instructions into a much larger conversation history.

Q: When should an agent.md or claude.md file be used?

A persistent agent file should be used when the model needs proprietary company information or a specific methodology during every conversation and every turn. The speaker argues that most people do not have this requirement. General facts that the agent can discover from the codebase, such as whether a project uses React, do not need to be repeated in an always-loaded instruction file.

Q: Why can large persistent instruction files waste tokens?

Persistent instruction files are inserted into the context repeatedly as the conversation continues. The example describes a thousand-line claude.md file as potentially containing about 7,000 tokens, all of which may be spent on every run. If those instructions apply only to occasional tasks, this repetition uses context that could otherwise hold the current request, project material, tool information, and useful conversation history.

Q: How do AI agent skills use progressive disclosure?

Progressive disclosure means that the agent initially receives only a skill's name and description. When a request matches that description, the agent can open the full skill document and use its detailed procedure. For example, a request to create a Notion report could trigger a matching report skill. Unrelated tasks would not load the skill's full instructions, conserving context and tokens.

Q: How should you create a reliable AI agent skill?

Start by completing the intended workflow with the agent rather than immediately writing a skill file. Guide the agent through each step, inspect its decisions, identify failures, and provide the missing criteria or corrections. Once the workflow produces the desired result consistently, capture that tested process as a skill. This approach turns practical feedback and observed mistakes into explicit, reusable operating instructions.

Q: Why did the sponsor research agent approve every company?

The sponsor research request was too broad. The agent was told to check incoming emails, research each sponsor, and decide whether the opportunity was worthwhile, but it was not given a sufficiently detailed evaluation process. It consequently marked every sponsor as legitimate. Stronger results appeared after the workflow required checks of Twitter, YouTube, Trustpilot, and funding, plus an automatic rejection rule based on missing or poor indicators.

Q: How can an AI agent workflow be improved after weak results?

A weak workflow can be improved by examining exactly which checks the agent skipped and then walking through those checks together. In the sponsor example, the user asked the agent to research specific sources, verify the company's standing and funding, and apply an explicit rejection condition. The corrected decisions could then be recorded in a Google Sheet, creating a clearer process that could later become a reusable skill.

Summary & Key Takeaways

• Modern AI models are already highly capable, so output quality increasingly depends on how their context is assembled. An agent may receive a provider system prompt, persistent instruction files, skill descriptions, tool definitions, a codebase, and conversation history. Unnecessary material consumes tokens and can reduce the space available for useful task information.

• Persistent files such as agent.md or claude.md are loaded on every turn, making them appropriate mainly for proprietary information or essential methods that must always be present. Skills are more efficient for specialized workflows because only their names and descriptions remain visible until the agent identifies a relevant task and opens the complete instructions.

• Reliable skills should emerge from practical collaboration with the agent. Run the workflow step by step, inspect weak decisions, supply missing criteria, and repeat the process until the desired result is consistent. The sponsor-screening example improved only after explicit checks and rejection rules replaced a vague request to research each company.