Pulmonary Health Case Study: Bias Exploration, Exploring Fairness in Machine Learning

TL;DR

This video explores bias in machine learning models through a case study on pulmonary health diagnostics, finding that representativeness across protected variables does not significantly affect model accuracy.

Transcript

[MUSIC PLAYING] AMIT GANDHI: Hi, my name is Amit Gandhi. And I'm a graduate researcher at MIT. Welcome to this course on exploring fairness in machine learning for international development. In this video, we will examine bias in machine learning models through a pulmonary health diagnostic case study. In particular, we will explore the influence o... Read More

Key Insights

• 🔨 Pulmonary diseases in remote areas with limited access to healthcare often go undiagnosed and untreated, motivating the development of a screening tool.
• 🎰 Introducing imbalances in protected variables (gender and income) did not significantly affect the accuracy of the machine learning model for pulmonary disease prediction.
• 🥺 Gender imbalance was correlated with smoking in the data, leading to higher predictive accuracy in women for COPD due to a more homogeneous population.
• 😘 COPD was the most sensitive to socioeconomic status, showing a 4% difference in model accuracy between high and low-income populations.

Questions & Answers

Q: What was the motivation behind developing a screening tool for pulmonary diseases?

The motivation was to provide a tool for community health workers in remote areas to determine if patients presenting symptoms actually have pulmonary diseases since these conditions often go undetected and untreated.

Q: How was the data for the study collected?

Data was collected from 303 patients who sought medical care at health clinics in Kuna, India. Two exams were administered, a mobile health diagnostic kit developed by Dr. Fletcher's group and measurements from a pulmonary function test lab.

Q: What variables were considered in the bias study?

The study explored biases related to gender and income. The population distributions for these variables were analyzed to examine the influence of representativeness on model accuracy.

Q: Did the study find any significant decrease in accuracy as gender imbalances were introduced in the data?

Surprisingly, no significant decrease in accuracy was observed as gender imbalances were introduced. The lack of representativeness in protected variables may not always introduce bias or fairness into models.

Summary & Key Takeaways

• The video discusses the development of a screening tool for pulmonary diseases in remote areas with limited access to healthcare.

• Data was collected from 303 patients in Kuna, India, and the influence of representative data on accuracy was examined.

• The study found that introducing imbalances along protected variables (gender and income) did not significantly impact the accuracy of the machine learning model.