How Can Claude Learn Workflows by Watching You?

TL;DR

Claude can turn a recorded work demonstration into a reusable skill by observing screen activity, clicks, typing, and optional spoken reasoning. The strongest results come from narrating decisions that the interface cannot reveal, then testing the generated instructions on new material to see whether Claude learned the underlying judgment rather than merely copying visible actions.

Transcript

Claude can now learn how you work just by watching you do it. You do the task, you explain the decisions you're making, and Claude can create the instructions by itself. So, I gave mine a completely new version of the task after doing exactly this. And that's when this got interesting because for the last few years, getting better results from AI m... Read More

Key Insights

• Claude's skill recorder captures screen activity, clicks, typing, and optional microphone narration, then converts the observed demonstration into reusable instructions for repeating the task.
• Spoken narration supplies reasoning that Claude cannot infer from visible actions alone, such as why one metric matters, why another is ignored, or why a particular item should appear first.
• The weekly-update workflow selects actionable changes from four sources and organizes them into what happened, what is on track, what is at risk, and what needs a decision.
• Repeated and blocking support issues deserve attention in the demonstrated workflow, while isolated complaints are excluded because they do not establish a pattern or materially prevent customer progress.
• Effective summarization is a judgment task because the difficult step is choosing which information deserves to remain, not merely rewriting a long document with fewer words.
• Urgency and decision relevance determine prominence in the executive summary, so delayed training leads while completed migration work receives only one line despite its large numbers.
• Acceptable or settled items receive little space because they require no action, while unresolved decisions, future staffing risks, and issues capable of disrupting later milestones remain visible.
• Transfer testing reveals whether Claude learned a preference rather than copied an example, because the generated skill must process completely different source material and still make similar prioritization choices.
• More videos from this channel:
  • How to Launch an AI-Assisted Rental Business
  • Which Free AI Courses Can Help You Get Hired?
  • How to Become Irreplaceable in the AI Economy
• More from this channel: AI Founders

Questions & Answers

Q: How does Claude learn a workflow by watching someone work?

Claude records the user's screen activity, clicks, and typing while the task is performed. If the microphone is enabled, it also captures the user's spoken explanation of decisions. After the demonstration, Claude reviews what happened and turns the observed process into reusable instructions. Those instructions can then be run on new source material to test whether the workflow transfers.

Q: Why should users narrate their decisions while recording a Claude skill?

Narration communicates reasoning that cannot be observed from the screen alone. Claude may see that a sentence was deleted or a metric was selected, but it cannot reliably know why. Explaining the choice at the moment it is made reveals distinctions such as actionability, urgency, repetition, normal variation, or future risk, making the resulting instructions more faithful to the user's process.

Q: What information belongs in the demonstrated weekly update?

The weekly update includes sales changes that affect the current week, support problems that are repeated or block a customer, project items whose status changed since the prior week, and meeting decisions or unresolved questions. It excludes totals that do not change Monday's actions, isolated complaints, unchanged project items, and detailed records of who said what during meetings.

Q: How should Claude organize a recurring business update?

The demonstrated workflow organizes selected information into four stable sections: what happened, what is on track, what is at risk, and what needs a decision. Source material comes from sales, support, the project tracker, and meeting notes. The organization remains consistent each week, while the included details change according to actionability, status movement, recurring problems, and unresolved choices.

Q: How can Claude learn a person's summarization preferences?

Claude can observe which passages a person keeps, shortens, prioritizes, or removes while reducing a long project update. Spoken reasoning clarifies the preference behind each choice. In the demonstration, delayed training is placed first, completed migration gets one line, acceptable vendor performance disappears, and unresolved decisions remain because the recipient needs to act on them.

Q: Why is summarization more than making a document shorter?

Summarization requires deciding what deserves to survive. A large completed item may be less important than a smaller unresolved risk because finished work needs no decision. The demonstrated approach evaluates whether information changes what readers should do next, deserves prominent placement, prevents unnecessary questions, threatens a future milestone, or can be removed without affecting action.

Q: How can you test whether a recorded Claude skill transfers to new work?

Give the generated skill completely new source material and compare its result with a version prepared independently by the person who demonstrated the task. Matching every sentence is not necessary. The useful test is whether Claude selects similar categories of information, applies comparable priorities, preserves the intended format, and handles different facts according to the reasoning shown during recording.

Q: What are the limits of learning professional judgment from one demonstration?

A single demonstration can produce a close result, but the transcript shows that some details still need adjustment and that subjective outputs may differ. Standard sources and formats are easier than preference-heavy work. The harder test is whether Claude can notice meaningful irregularities and handle material it was never directly shown, which distinguishes remembering actions from understanding their intended logic.

Summary & Key Takeaways

• Claude's skill recorder captures a task as it is performed, including visible screen activity, clicks, typed input, and optional microphone narration. Claude then reviews the demonstration and creates reusable instructions. The approach replaces the difficult job of documenting an entire workflow beforehand with performing the work once while explaining key decisions.

• The first experiment turns information from sales, support, a project tracker, and meeting notes into a weekly update. The demonstration teaches Claude which details affect action, which repeated or blocking problems matter, which status changes belong, and how decisions and open questions should fit into four consistent report sections.

• Later experiments test whether demonstrations can transfer subjective preferences and professional judgment. Claude watches a long update being reduced according to urgency, decision relevance, and potential future impact. The final challenge asks whether it can apply the intended reasoning to unfamiliar material, rather than simply reproduce actions or memorize examples.