What is the False Discovery Rate in Statistics?

TL;DR

The false discovery rate (FDR) measures the proportion of false positives among rejected hypotheses, allowing researchers to manage mistakes in large testing scenarios. By accepting some false discoveries, the FDR facilitates more findings, particularly in scientific research, where exploring promising leads can outweigh the risks of errors.

Transcript

all right so that's it for the family-wise error rate and now we're going to move on to a topic called the false discovery rate which is sort of a more contemporary take on multiple testing so it really goes back to this table where remember this corresponds to the possible outcomes for m hypothesis tests that we've conducted and we can control for... Read More

Key Insights

• 👻 FDR is a contemporary approach to multiple testing that allows for a controlled number of false positives.
• ⚖️ FWER is concerned with avoiding all false rejections, while FDR balances the trade-off between false positives and discoveries.
• 👨‍🔬 FDR is particularly useful in settings with a large number of tests, such as scientific research with many drug targets.
• 🎮 The Benjamini-Hochberg procedure provides a method to control the FDR.
• 👻 The procedure allows for a higher number of false positives in exchange for more discoveries.
• ✋ FDR should not be used in contexts where the cost of false positives is high, such as criminal justice cases.
• 🧑‍💼 FDR is a statistical measure that requires thoughtful consideration of the trade-offs involved.

Questions & Answers

Q: What is the difference between the family-wise error rate (FWER) and the false discovery rate (FDR)?

FWER aims to minimize the probability of falsely rejecting any null hypothesis, while FDR focuses on the fraction of rejected null hypotheses that were actually true. FWER prevents any type 1 errors, while FDR allows for a controlled number of false positives.

Q: In which situations is FDR more useful than FWER?

FDR is more useful in settings with a large number of tests where it may be impractical to avoid all type 1 errors. For example, in scientific research with thousands of drug targets, FDR allows for more discoveries while still controlling the number of false positives.

Q: How is the false discovery rate (FDR) computed?

The FDR is computed as the expectation of the ratio of false rejections to the total number of rejections. It represents the fraction of rejected null hypotheses that were actually false discoveries.

Q: What is the purpose of the Benjamini-Hochberg procedure?

The Benjamini-Hochberg procedure is a method to control the FDR. It involves specifying a threshold level (q), ordering the p-values, determining the largest index (l) where the p-value is less than q times j/m, and rejecting all null hypotheses with p-values smaller than the l-th smallest p-value.

Summary & Key Takeaways

• The family-wise error rate (FWER) focuses on the probability of falsely rejecting any null hypothesis, while the FDR focuses on the fraction of rejected null hypotheses that were actually true.

• While FWER aims to minimize type 1 errors, FDR allows for a controlled number of false positives in exchange for a higher number of discoveries.

• The FDR is useful in settings with a large number of tests, such as scientific research, where having some false positives is acceptable as long as discoveries are made.