What is the best mix of humans and AI for making decisions

TL;DR

Augmented intelligence, where humans collaborate with AI, often yields the highest decision accuracy, especially for mid to high confidence predictions. The best approach uses AI for fast, confident cases and human judgment for uncertain or complex ones, while presenting AI input in a way that minimizes bias. This partnership reduces workload and improves outcomes when implemented carefully.

Transcript

A decision needs to be made. But who should make it? Me, a human, ... or an artificial intelligence, an AI? We've discussed before that humans can outperform AI at some tasks, but that, statistically, AI will make a better job of deciding for other tasks. So for one single decision, who should decide? Well, the answer is a fascinating combination o... Read More

Key Insights

• AI performance curves show high accuracy when confidence is very high and very low accuracy when confidence is uncertain.
• Humans can outperform AI at mid confidence levels due to context and the ability to look things up or consult colleagues.
• Augmented intelligence sits between AI and human performance, offering the highest overall success rates for many predictions.
• Forced display of AI input leads to automation bias where humans rely too much on AI recommendations.
• Optional display reduces bias by letting humans form a first impression before consulting AI.
• Trust issues arise when AI shows accuracy percentages; people may resist ignoring a displayed recommendation.
• Human cognitive bias must be accounted for to maximize the benefits of AI augmentation.
• A structured approach to decision making uses both human insight and AI while mitigating bias and distractions.
• More videos from this channel:
  • How to connect AI agents to tools with MCP and vaults
  • How to choose between long context and CAG in LLMs
  • What is the difference between Observability, Monitoring, APM
• More from this channel: IBM Technology

Questions & Answers

Q: What is augmented intelligence and why is it beneficial for decision making

Augmented intelligence is the combination of human decision making aided by AI recommendations. It falls between the two extremes of AI alone and human alone. The video argues that for a significant portion of predictions, especially mid to high confidence cases, augmentation yields the highest success rate. It leverages AI speed and pattern recognition with human context and judgment to handle complex or rare situations. This approach can reduce workload while improving accuracy, provided that information presentation is designed to minimize cognitive biases and avoid automation bias.

Q: How do confidence levels affect whether AI or humans should decide

Confidence levels determine who should decide by indicating how reliable an AI prediction is. When AI is highly confident, it tends to outperform humans. When AI is unsure, humans can add value by applying additional information and context. A 50 percent confidence level is a point where humans often perform better than AI. The video emphasizes using a mixed approach to maximize accuracy across different confidence scenarios.

Q: What is automation bias and how can it be avoided

Automation bias is the tendency for humans to favor automated recommendations and ignore contradictory information. It occurs when decisions are forced to include AI input. The video suggests optional display, where the AI recommendation is shown only when requested, to give humans a chance to form their own judgment first and reduce bias. This helps preserve human autonomy and improves decision quality.

Q: What are forced display and optional display models

Forced display shows AI recommendations alongside every case, which can lead to automation bias as analysts may rely on AI judgment. Optional display presents the AI input only if the human asks for it, allowing first impressions to form independently. The video argues that optional display helps mitigate bias and improves trust and decision quality by balancing AI support with human judgment.

Q: Why is reliability of AI input important in decision making

Accurate AI input with transparent explanations can influence human decisions. When AI shows high accuracy, people may trust it more, but if they see low accuracy or uncertain results, trust can drop. The video notes that trust is affected by how the AI presents its recommendations and accuracy, and that presenting uncertainty carefully is crucial to avoid overreliance or dismissiveness.

Q: How does human context improve AI decisions

Humans bring context, knowledge, and the ability to look things up or consult colleagues, which can help in complex or statistically rare cases where AI alone may falter. In mid confidence regions, humans often outperform AI by applying broader information and reasoning. This human advantage is a key reason for combining AI with human judgment rather than relying solely on machines.

Q: What role does presenting information play in AI-assisted decisions

Presentation of AI input significantly influences how information is used. The right presentation can enhance decision quality by aligning AI recommendations with human reasoning and reducing cognitive biases. The video emphasizes designing the display to minimize automation bias and to support a collaborative interaction between human decision makers and AI algorithms.

Q: What is the overall takeaway about who should decide

The video argues that the best decision making is often a combination of humans and AI, known as augmented intelligence. It emphasizes that the most effective approach depends on the confidence level of AI predictions and that for many cases, augmentation yields the highest success rate. The key is to structure the workflow and presentation to minimize bias and maximize accurate collaboration.

Summary & Key Takeaways

• A hybrid approach combines AI and human input to improve decision outcomes and reduce bias. The video explains how confidence levels influence whether AI alone, human input, or augmentation yields the best results. It also covers how display methods affect automation bias and trust in AI recommendations.

• The key takeaway is that augmented intelligence can outperform either humans or AI alone, but only when the presentation of AI input is managed to minimize cognitive bias and when confidence-aware allocation is used.

• The discussion uses a fraud detection example to illustrate decision curves, the role of human context, and the importance of avoiding overreliance on AI in uncertain cases.