8. Parametric Hypothesis Testing (cont.)

TL;DR

Tests are used to make binary decisions based on statistical evidence, with the p-value serving as a measure of this evidence.

Transcript

The following content is provided under a Creative Commons license. Your support will help MIT OpenCourseWare continue to offer high-quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu. PHILIPPE RIGOLLET: We're talking about tests. And to be... Read More

Key Insights

• ❓ Hypothesis testing involves comparing a null hypothesis (h0) to an alternative hypothesis (h1) to make a binary decision.
• 🏆 Tests involve dividing data into two hypotheses and using a test statistic to map the data to a binary decision.
• 🅰️ Type I and type II errors represent the two types of mistakes one can make in hypothesis testing.
• 😀 The p-value is a measure of evidence against the null hypothesis and is the probability of observing data as extreme or more extreme than what we observed, assuming the null hypothesis is true.

Questions & Answers

Q: What is the role of the null hypothesis in hypothesis testing?

The null hypothesis, denoted h0, represents the status quo and is the assumption to be tested against the alternative hypothesis h1. Its role is to provide a baseline against which we compare evidence in the data.

Q: Why are type I and type II errors important in hypothesis testing?

Type I and type II errors are important because they represent the two types of mistakes we can make when conducting hypothesis tests. Type I errors occur when we reject the null hypothesis even though it is true, while type II errors occur when we fail to reject the null hypothesis even though it is false. Understanding and controlling these error rates are crucial in hypothesis testing to ensure the validity of our conclusions.

Q: What is the power of a test?

The power of a test, denoted pi of psi, is the probability that the test correctly rejects the null hypothesis when the alternative is true. In other words, it measures the ability of the test to detect a true effect or relationship. A higher power is desired as it means a higher chance of recognizing the alternative hypothesis when it is in fact true.

Q: How does one compute a p-value?

The p-value can be computed by comparing the test statistic (e.g., t or z-statistic) to its distribution under the null hypothesis. More specifically, the p-value is the probability of observing data as extreme or more extreme than what we observed, assuming the null hypothesis is true. This can be calculated using the cumulative distribution function (CDF) of the test statistic's distribution.

Summary & Key Takeaways

• Tests involve dividing data into two hypotheses: h0 (status quo) and h1 (alternative).

• A test uses a statistic to map data into a binary decision (0 or 1) and has a rejection region (set of data points that lead to rejecting h0).

• Tests have two types of errors: type I (rejecting h0 when it is true) and type II (failed to reject h0 when h1 is true).

• The p-value is the smallest level at which a test rejects h0 and provides a measure of statistical evidence against h0.

• The Neyman-Pearson paradigm fixes the level of type I error (alpha) and aims to minimize type II error under that constraint.

• P-values provide a measure of statistical evidence against h0 and can be thresholded to make decisions.