Statistical Learning: 13.Py Multiple Testing I 2023

TL;DR

This content focuses on multiple testing, specifically the familywise error rate and false discovery rate. It discusses hypothesis testing, P-values, and methods for controlling the rate of false positives in multiple comparisons.

Transcript

welcome back uh today we're going to talk uh we're going to go through the lab for chapter 13 uh multiple testing uh and we're going to focus on um two main Criterion for multiple testing familywise error rate and false Discovery rate um as well as how to estimate things like false Discovery rate okay so as usual we have our um our standard Imports... Read More

Key Insights

• 💄 Multiple testing involves making simultaneous comparisons between multiple hypotheses.
• ☠️ The familywise error rate measures the probability of making any false positives in multiple testing.
• ☠️ The false discovery rate considers the proportion of false positives among the rejected hypotheses.
• 💻 Adjusted P-values can be computed using methods like the Bonferroni correction or Hochberg's method.
• ☠️ Controlling the familywise error rate is crucial to minimize the chances of false discoveries.
• #️⃣ The number of false positives and false discovery proportion can be influenced by the threshold chosen for significance.
• ☠️ Methods like the Hochberg's method can be used to adjust P-values and control the rate of false positives in multiple testing.

Questions & Answers

Q: What are the two main criteria for multiple testing?

The two main criteria for multiple testing are the familywise error rate and the false discovery rate. The familywise error rate measures the overall probability of making any false positives, while the false discovery rate measures the proportion of false discoveries among the rejected hypotheses.

Q: How can adjusted P-values be computed for multiple tests?

Adjusted P-values can be computed using methods like the Bonferroni correction or the Hochberg's method. The Bonferroni correction involves multiplying each individual P-value by the number of tests, while Hochberg's method adjusts the P-values in a step-up procedure based on the order of significance.

Q: What is the importance of controlling the familywise error rate in multiple testing?

Controlling the familywise error rate is important to reduce the probability of making any false positives. By setting a strict threshold for significance, researchers can minimize the chances of incorrectly rejecting the null hypothesis.

Q: How can the false discovery proportion be calculated in multiple testing?

The false discovery proportion can be calculated by dividing the number of false discoveries (incorrectly rejected null hypotheses) by the total number of rejected hypotheses. It provides a measure of the proportion of false positives among the significant results.

Summary & Key Takeaways

• The content covers the concepts of multiple testing, familywise error rate, and false discovery rate.

• It explores hypothesis testing using T-tests and calculates P-values for each test.

• The importance of controlling the rate of false positives and false discovery proportion is highlighted.