Statistical Learning: 13.4 Holm's Method for Controlling FWER

TL;DR

The Holm method, while more complex, can often reject more null hypotheses and make more discoveries compared to the Bonferroni method, although both methods control the family-wise error rate.

Transcript

okay so i'm going to talk about one other method that's a little bit superior to bonfroni in some ways and uh and but it's a little more complicated so it's not quite as popular i think but let me quickly go through how this approach works so this is known as holmes method and it starts the same way as bonferroni you just perform standard hypothese... Read More

Key Insights

• ⚾ The Holm method starts similarly to the Bonferroni method but applies a different algorithm to determine the significance of p-values based on stricter thresholds.
• 🥺 Holm method can reject null hypotheses that are not rejected by the Bonferroni method, leading to more discoveries.
• ❓ The Bonferroni method is sometimes considered overly conservative, potentially rejecting fewer hypotheses than warranted by the data.
• ❓ Choosing between the Bonferroni and Holm methods depends on the preference for simplicity (Bonferroni) or maximizing rejections (Holm).
• ☠️ The Holm method offers a more nuanced approach to controlling the family-wise error rate, making it a viable alternative to the Bonferroni method.
• 😀 A comparison of p-values under both methods reveals instances where the Holm method rejects more hypotheses than the Bonferroni method.
• ✋ In extreme cases, the Holm method can significantly outperform the Bonferroni method, rejecting a much higher number of null hypotheses.

Questions & Answers

Q: How does the Holm method differ from the Bonferroni method in controlling the family-wise error rate?

The Holm method involves ordering p-values and comparing them to decreasing thresholds, while the Bonferroni method uses a fixed threshold for all hypotheses. Holm method can more efficiently reject null hypotheses.

Q: Why is the Holm method considered superior to the Bonferroni method in some cases?

The Holm method often rejects more null hypotheses compared to the Bonferroni method, allowing for more discoveries while still controlling the family-wise error rate. It can be a better choice when maximizing rejections is desired.

Q: Are both the Bonferroni and Holm methods guaranteed to control the family-wise error rate?

Yes, both methods ensure that the family-wise error rate remains below a specified threshold, typically 0.05. They provide statistical safeguards against making too many false discoveries.

Q: Which method is more popular, Bonferroni or Holm?

Bonferroni is currently more popular, with about 99% of researchers and analysts using it due to its simplicity. The Holm method, although slightly more complex, is gaining recognition for its potential to reject more hypotheses.

Summary & Key Takeaways

• The Holm method is an alternative to the Bonferroni method for controlling the family-wise error rate in hypothesis testing.

• Both methods involve ordering p-values and comparing them to specific thresholds based on alpha and the number of hypotheses.

• Holm method may reject more null hypotheses than Bonferroni method, allowing for more discoveries.