Statistical significance on bus speeds | Probability and Statistics | Khan Academy

TL;DR

A woman conducts a randomized experiment to compare the travel times of two buses and finds that Bus A is, on average, 8 minutes faster than Bus B.

Transcript

• [Voiceover] "Giovanna usually takes bus B to work, "but now she thinks that bus A gets her to work faster. "She randomized 50 workdays between a treatment group "and a control group. "For each day from the treatment group, she took bus A; "and for each day from the control group, she took bus B. "Each day she timed the length of her drive." This ... Read More

Key Insights

• ❓ Randomization is important in experiments to ensure unbiased and reliable results.
• 💄 Human tendency is to unintentionally favor certain circumstances or choices, making randomization crucial in avoiding biased outcomes.
• 🏆 Re-randomization can be used to test the significance of observed results and determine the probability of obtaining those results by chance.

Questions & Answers

Q: Why did Giovanna randomize the 50 workdays between the treatment and control groups?

Giovanna randomized the workdays to ensure that her results were not influenced by any factors other than the buses being compared. By randomizing, she eliminated the possibility of bias or confounding variables affecting the results.

Q: How did Giovanna ensure the randomization process was truly random?

Giovanna used a method where she wrote down each workday on a piece of paper, closed her eyes, and randomly moved them to the left or right side of her table. This ensured that the assignment of each workday to either Bus A or Bus B was completely random.

Q: What is the significance of the 9.3% result she obtained from re-randomizing the data?

The 9.3% result represents the probability of obtaining a result as extreme as or more extreme than the actual experiment purely by chance. Since this is higher than the typical threshold of 5% for statistical significance, it suggests that the observed difference in travel times between the buses may not be statistically significant.

Q: Does Giovanna's experiment prove that Bus A is faster than Bus B?

The experiment provides evidence that Bus A is, on average, 8 minutes faster than Bus B. However, statistical significance is not fully established, as the observed result could still be due to random chance. Further research with a larger sample size may be needed to confirm the findings.

Summary & Key Takeaways

• Giovanna randomized 50 workdays between a treatment group (using Bus A) and a control group (using Bus B) to compare their travel times.

• The experiment showed that Bus A has a median travel duration 8 minutes less than Bus B.

• To test the results, Giovanna re-randomized the data 1000 times and found that 9.3% of the randomizations resulted in a median travel duration for Bus A that was 8 minutes less than Bus B.