How AI Agents Speed IT Anomaly Resolution with Context Curation

TL;DR

Agentic AI speeds anomaly resolution by guiding SREs with a curated set of signals, validated root causes, and actionable runbooks, reducing MTTR. It avoids feeding every log into a model to prevent hallucinations, instead using topology aware correlation to pull only relevant data and provide explainable reasoning for human oversight.

Transcript

Did you know that if I woke you up and then asked  you to work on a problem like well resolving an IT system anomaly, it would take you on average  something like 22 minutes to move from a sleeping state to a cognitive state where you're like  actually productive? And with every minute of downtime costing potentially thousands of dollars,  that can... Read More

Key Insights

• AI agents use context curation to limit data fed to models, which reduces hallucinations and improves accuracy.
• Topological correlation helps agents fetch only data from components involved in the incident, avoiding unrelated signals.
• The agentic process follows perception, reasoning, action, and observation in a feedback loop to refine hypotheses.
• Probable root causes are identified with explainability, showing the reasoning chain and supporting evidence for human review.
• Validation steps are generated by the AI to help SREs confirm the identified root cause before remediation.
• Runbooks provide a concrete, ordered set of remediation steps that can be executed quickly by the SRE team.
• Automation scripts and workflows translate runbook steps into executable commands, increasing operational speed and consistency.
• Documentation generation is an automatic byproduct, creating post incident reports and ongoing incident summaries.
• More videos from this channel:
  • How to Govern AI Agents with Five Core Pillars
  • How Do Large Language Models Work and Apply?
  • How does MCP differ from API for AI data access
• More from this channel: IBM Technology

Questions & Answers

Q: How can AI agents prevent hallucinations when diagnosing IT incidents

AI agents prevent hallucinations by using context curation, which means they do not dump all telemetry data into a model. Instead they rely on topology aware correlation to pull only the signals that matter for the current incident. This focused data set helps the agent form a plausible hypothesis while avoiding coincidental patterns that do not reflect the actual root cause. The agent also provides explainability so human operators can see the reasoning path and evidence supporting the proposed root cause, enabling validation before any remediation actions are taken.

Q: What is the role of topology aware correlation in anomaly resolution

Topology aware correlation plays a central role by leveraging the dependency map of services to fetch telemetry only from the components involved in the incident. This means that when an authentication service has issues, the agent will check related components like the user database, caches, and recent deployments rather than reviewing data from unrelated services. This focused approach reduces noise and accelerates accurate root cause identification.

Q: How does the agentic AI propose remediation steps

The agentic AI generates a step by step runbook once a probable root cause is identified. This runbook provides an ordered list of remediation steps that the SRE can follow quickly, even if they are not deeply familiar with the affected component. Steps might include actions like archiving old logs, restarting a service, and setting up monitoring thresholds to prevent recurrence, all with the intent of enabling fast, reliable remediation.

Q: What is the importance of explainability in AI-assisted incident response

Explainability is important because it makes the agent’s reasoning process transparent to the human operator. By showing the chain of thought and supporting evidence, the SRE can understand why the AI identified a certain root cause and assess whether the conclusions are credible. This transparency helps build trust, facilitates validation, and ensures that human oversight remains central in production remediation.

Q: How do automation scripts enhance the incident response workflow

Automation scripts convert the AI’s remediation steps into executable commands or configurations like bash scripts or Ansible playbooks. This allows the SRE to implement fixes quickly with precise syntax and parameters, increasing consistency and reducing manual error. Automation also enables faster replication of fixes in other environments and helps scale incident response across multiple incidents.

Q: What is the difference between detection and diagnosis in this context

Detection refers to identifying that an incident has occurred, such as a failed login or slow service, while diagnosis is about identifying the probable root cause. The described system operates in a post incident detection scenario, focusing on diagnosing what happened, determining the root cause with evidence, and then guiding remediation rather than predicting incidents before they occur.

Q: How does context curation affect data management during incidents

Context curation improves data management by filtering and prioritizing the most relevant telemetry for the current incident. It reduces the volume of data sent to AI models, mitigates the risk of noise, and helps the agent build a coherent and testable hypothesis. This approach supports faster, more accurate resolution and reduces cognitive load on the SRE.

Q: What outcomes are expected from using agentic AI in IT operations

Expected outcomes include faster anomaly detection and resolution, reduced MTTR, lower operational stress, and less sleep inertia during off hours. The system emphasizes human oversight, validated root causes, and automated runbooks, along with automatic documentation and ongoing incident summaries to improve collaboration and reliability across IT operations.

Summary & Key Takeaways

• Agentic AI relies on context curation to filter signals before analysis, preventing hallucinations from noisy data and improving diagnostic accuracy. The approach uses a dependency graph to pull only relevant metrics, events, and logs for each incident.

• The workflow follows perception, reasoning, action, and observation in a loop, enabling iterative hypothesis formation and evidence gathering to identify probable root causes.

• The system emphasizes explainability and human oversight, showing the chain of thought and supporting evidence so SREs can validate conclusions before remediation.

• Agents aid validation by proposing verification steps, generate step by step runbooks, and can translate those steps into automation scripts while documenting the incident.

• Runbooks provide ordered remediation steps, such as archiving logs, restarting services, and monitoring usage to prevent recurrence.

• Automation scripts and workflows turn recommendations into executable actions, reducing manual effort and speeding up the resolution process.

• Post incident, the AI can draft incident reports and ongoing summaries to keep new responders aligned, improving collaboration and reducing downtime.

• Overall, AI agents augment human decision making in IT operations, aiming to cut downtime and ease the workload on night shifts.