A Roadmap for the Rigorous Science of Interpretability | Finale Doshi-Velez | Talks at Google

TL;DR

The speaker discusses the importance of interpretability in machine learning models and proposes a spectrum for evaluating interpretability, ranging from qualitative to quantitative measures. They also suggest that the cognitive science of explanation should be considered when evaluating interpretability.

Transcript

[MUSIC PLAYING] SPEAKER: We are excited to have our speaker, Finale Doshi-Velez, here today. She is an assistant professor in computer science at the Harvard Paulson School of Engineering and Applied Sciences. Her research focuses on the intersection of machine learning and health care. And today, she'll be speaking to us about interpretability tow... Read More

Key Insights

• 👻 Interpretability is important in machine learning as it allows humans to understand and trust the decisions made by the model.
• 🌍 Evaluating interpretability requires a spectrum of measures, including qualitative assessments, real-world application performance, and cognitive science insights.
• 💁 Different domains may have different needs for interpretability, and there may not be a single form of explanation that is universally interpretable.

Questions & Answers

Q: What is the definition of interpretability in machine learning?

Interpretability in machine learning refers to the ability of a model to provide meaningful explanations for its decisions and predictions. It involves understanding the internal workings of the model and the factors that contribute to its output.

Q: How can interpretability be measured and evaluated in machine learning models?

Interpretability can be evaluated through qualitative and quantitative measures. Qualitative evaluation involves assessing whether a model's explanation is understandable to humans and whether it helps in decision-making. Quantitative evaluation can include measures like sparsity, accuracy, or task performance.

Q: What are the different reasons why interpretability is needed in machine learning?

Interpretability is needed to debug systems, ensure safety, identify mismatched objectives, and meet legal requirements. It is also important for scientific discovery, understanding complex data, and gaining insights into the operation of machine learning systems.

Q: Is there an inverse relationship between model performance and interpretability?

The relationship between model performance and interpretability is not necessarily inverse. While some simpler models may be more interpretable, deep models can also be interpretable if they provide explanations at a local level or in terms of cognitive chunks. The trade-off between model performance and interpretability can be context-specific.

Summary & Key Takeaways

• The speaker emphasizes the need for interpretability in machine learning systems, especially as they become more widely deployed.

• They discuss the different aspects of interpretability and how it can be measured and evaluated.

• The speaker also introduces the idea of evaluating interpretability in the context of real applications, as well as the importance of considering the cognitive science behind explanation.