StatQuickie: Thresholds for Significance

TL;DR

Understanding significance thresholds like 0.05 and their implications for data analysis and publishing.

Transcript

hello and welcome to the very first stat quickie stat quickies are a little short video it's not a full-on stat quest where it's only gonna take a few minutes where we address a question that someone's asked me in the comments or sent me an email or if I just ran into someone in the hallway and they said hey I got a question so that's what's that q... Read More

Key Insights

• ❓ Significance thresholds like 0.05 are common but arbitrarily chosen in statistics.
• 😘 Lower significance thresholds can be beneficial with substantial effect sizes.
• 📶 Effect size is crucial as it indicates the strength of the relationship between variables.
• 🤯 Extraordinary claims necessitate extraordinary evidence with extremely low p-values.
• 😀 Explore data freely with less emphasis on p-values for preliminary findings.
• 😀 Balanced consideration of p-values and effect sizes is essential for meaningful statistical analysis.
• 👨‍🔬 Selecting appropriate significance thresholds depends on the research context and the strength of the relationships.

Questions & Answers

Q: Why was the significance threshold of 0.05 chosen for statistics?

The threshold of 0.05 was not based on natural reasons but rather as a convention to balance statistical errors and significance levels in research.

Q: When is it appropriate to use a significance threshold lower than 0.05?

Lower thresholds are beneficial when the effect size is substantial and can explain a significant portion of the data, indicating strong correlations.

Q: Why does the effect size matter in addition to the p-value?

Even with a low p-value, if the effect size is small, the significance may not be meaningful as it doesn't adequately explain the variations in the data.

Q: What is the importance of having extraordinary data for extraordinary claims?

Extraordinary claims, like the existence of extraterrestrials, require exceptionally strong evidence, including extremely low p-values, to support such groundbreaking assertions.

Summary & Key Takeaways

• The significance threshold of 0.05 in statistics is commonly used in scientific publications but was arbitrarily chosen.

• Using a lower significance threshold can be beneficial if the effect size is substantial and can explain the data.

• Extraordinary claims require extraordinary data, necessitating extremely low p-values for exceptional findings.