Joseph Blitzstein: "The Soul of Statistics" | Harvard Thinks Big 4

TL;DR

Joseph Blitzstein is a Professor of the Practice of Statistics known for teaching Harvard’s Stat 110, Introduction to Probability, and for presenting The Soul of Statistics at Harvard Thinks Big 4. He argues that conditioning is the soul of statistics because every probability depends on available information. His bomber example shows how selection bias hides crucial missing data, making the full explanation worth reading.

Transcript

SPEAKER: Professor Blitzstein is a Professor of The Practice of Statistics. He is known for teaching the popular class, Stat 110, introduction to probability, which holds over 300 students each fall. He also has over 200,000 subscribers to the class on iTunes U. His research interests focus on statistical inference for complex networks. Professor B... Read More

Key Insights

• Conditioning is the soul of statistics, meaning every probability is conditional on the information you have, and conditional probability tells you how to update beliefs based on what you observe.
• Selection bias occurs when the data you get to see is not what you actually care about, as with bomber planes: you only observe the ones that survived and returned, not the ones shot down.
• Abraham Wald advised putting armor on the parts of returning WWII bombers that showed little or no damage, reasoning that planes hit in those spots did not make it back.
• Whenever you have a data set, you should think about how it was sampled, because ignoring the sampling process can lead to very misleading answers.
• Lombard's 1835 longevity study found chocolate makers lived to 73.6 years and professors to 66.6, but had only nine chocolate makers, raising a sample size issue.
• The 1835 study's finding that students lived an average of 20.2 years is an obvious conditioning error, since 20 is simply a normal age to be a student.
• Censoring is a statistical problem because we only know how long people lived after they died, so survival analysis exists to handle the unknown future lifespans of living people.
• Regression toward the mean means extreme observations tend to move toward the average on remeasurement, as Galton found that tall fathers had tall but less extreme sons.

Questions & Answers

Q: Who is Joseph Blitzstein?

Joseph Blitzstein is a Professor of the Practice of Statistics. He is known for teaching Stat 110, Introduction to Probability, a popular class with over 300 students each fall and over 200,000 subscribers on iTunes U.

Q: What does Joseph Blitzstein teach at Harvard?

Blitzstein teaches Stat 110, Introduction to Probability. The class holds over 300 students each fall and has over 200,000 subscribers on iTunes U.

Q: What are Joseph Blitzstein’s research interests?

His research interests focus on statistical inference for complex networks. He notes that network analysis can produce misleading answers when researchers ignore where a network came from and how it was sampled.

Q: What is the soul of statistics according to Joseph Blitzstein?

Blitzstein says conditioning is the soul of statistics. Conditional probability describes how beliefs should be updated using information that can be observed, and every probability is conditional on the information available.

Q: How did Abraham Wald decide where to put armor on World War II bombers?

Wald advised putting armor on areas showing little or no damage on the planes that returned. The returning planes had survived their existing bullet holes, while planes struck in the apparently undamaged areas may not have returned.

Q: What is selection bias in the bomber example?

The observed data came only from bombers that survived well enough to return. The real question concerned the missing planes, so using only damage patterns from surviving aircraft could point investigators toward the wrong areas.

Q: Why should researchers examine how a data set was sampled?

Sampling determines which information researchers can observe and which information remains missing. Blitzstein warns that ignoring sampling or failing to consider what the analysis conditions on can lead to very misleading answers.

Q: What problem did Blitzstein identify in Lombard’s 1835 longevity study?

Lombard reported that chocolate makers lived to 73.6 and professors to 66.6, but the chocolate-maker group contained only nine people. Blitzstein identifies this as a sample-size issue because the study included far more professors than chocolate makers.

Summary

Professor Blitznein, a professor of the practice of statistics, gives a talk titled "The Soul of Statistics." He begins by sharing a motivating example about British bombers during World War II and how statistician Abraham Wald advised them to put armor where the planes showed little or no damage. This example illustrates selection bias in statistics. Professor Blitznein emphasizes that conditioning is the soul of statistics, which means that all probabilities are conditional and our beliefs should be updated based on the information we have. He discusses the importance of understanding sampling and the danger of ignoring it, using the example of networks. Additionally, he explores selection bias and censoring in a longevity study and regression towards the mean in test scores. He concludes by discussing the conditional golden rule, which guides his teaching philosophy.

Questions & Answers

Q: What was the motivating example Professor Blitznein shared regarding British bombers during World War II?

Professor Blitznein shared the example of British bombers being shot down by the Nazis and how statistician Abraham Wald advised them on where to put armor. Instead of putting armor where the planes sustained heavy damage, Wald suggested putting armor where there was little or no damage on the planes that returned. This example illustrates the concept of selection bias in statistics.

Q: What does Professor Blitznein mean when he says "conditioning is the soul of statistics"?

By saying "conditioning is the soul of statistics," Professor Blitznein emphasizes that all probabilities are conditional on the information that we have. Conditional probability allows us to update our beliefs based on the observed information. It is a fundamental aspect of statistical thinking and analysis.

Q: How does Professor Blitznein highlight the importance of sampling in statistical studies?

Professor Blitznein discusses the danger of ignoring sampling in statistical studies, using the example of networks. He mentions that many studies on networks focus on analyzing the structure without considering how the network was sampled. This omission can lead to misleading answers and interpretations if researchers are not cautious about what they are conditioning on.

Q: What example does Professor Blitznein provide regarding selection bias and censoring?

Professor Blitznein shares a longevity study conducted in 1835. He presents average longevities for different professions, such as chocolate makers, professors, clocksmiths, locksmiths, and students. He explains the selection bias in the data set, such as the small sample size of chocolate makers and the issue of conditioning on the age of students. He also mentions censoring, which refers to the fact that we only know the lifespan of individuals after they have died.

Q: How does Professor Blitznein explain regression towards the mean?

Professor Blitznein explains regression towards the mean using the example of test scores. He describes how students who initially perform exceptionally well or poorly tend to move closer to the mean when they retake the test. He also mentions Sir Francis Galton's study on heights, which showed that tall fathers have sons who are not as tall on average, while short fathers have sons who tend to be taller. Regression towards the mean is a concept that applies to various domains and helps to maintain stability in populations.

Q: What is the conditional golden rule mentioned by Professor Blitznein?

The conditional golden rule is Professor Blitznein's teaching philosophy. It is based on the principle of "do unto others as you would have done unto you," but with the condition that it applies only to individuals who share the same interests, background, and knowledge. Professor Blitznein aims to follow this rule in his teaching and believes that conditional probability, although challenging, is worth contemplating.

Takeaways

Professor Blitznein emphasizes the importance of conditioning in statistics, highlighting its role in updating our beliefs based on observed information. He discusses the dangers of selection bias, ignoring sampling, and the misconception of regression towards the mean. Additionally, he shares his teaching philosophy, which is based on the conditional golden rule. Understanding these concepts is crucial for accurate statistical analysis and interpretation.

Summary & Key Takeaways

• Joseph Blitzstein argues that conditioning is the soul of statistics: every probability is conditional on the information we have. Conditional probability tells us how to update beliefs based on observed data, and this idea underlies all statistical thinking about what we can and cannot see.

• Abraham Wald's WWII bomber example illustrates selection bias. Rather than armoring where returning planes showed bullet holes, Wald advised armoring the undamaged areas, because planes hit there never returned. The data we observe is filtered by survival, so we must reason about the planes we cannot see.

• Lombard's 1835 longevity study and regression toward the mean show conditioning errors and sampling issues. Students appearing to live only 20.2 years reflects their age, not mortality. Galton found tall fathers have sons who regress toward the mean height, a phenomenon Kahneman connected to reward and punishment.