Examples of bias in surveys | Study design | AP Statistics | Khan Academy

TL;DR

The main sources of bias in polling methods are volunteer response sampling and convenience sampling.

Transcript

• [Instructor] We're told that David hosts a podcast and he is curious how much his listeners like his show. He decides to start with an online poll. He asks his listeners to visit his website and participate in the poll. The poll shows that 89% of about 200 respondents "love" his show. What is the most concerning source of bias in this scenario? A... Read More

Key Insights

• ❓ Volunteer response sampling is a biased method because participants choose to respond, and those who already have positive opinions are more likely to participate.
• 🥺 Convenience sampling, such as polling listeners who send fan emails, leads to overestimation of positive opinions due to a skewed sample.
• 💼 Response bias occurs when individuals do not answer truthfully, but in this case, the bias stems more from the sampling method rather than participant responses.
• 🏪 Undercoverage, the exclusion of important demographics, is not the most concerning source of bias compared to volunteer response and convenience sampling.
• 🛄 Polling methods should aim for random sampling to ensure representative results.
• ❓ Nonresponse, when individuals do not fill out surveys, can introduce bias if those who do not respond have different viewpoints that are not adequately represented.

Questions & Answers

Q: What is volunteer response sampling?

Volunteer response sampling occurs when participants choose to respond to a survey or poll. It is non-random and biased because people who are more likely to respond are those who already have a positive opinion of the subject, in this case, David's podcast.

Q: How does convenience sampling affect the accuracy of the poll results?

Convenience sampling refers to choosing participants based on accessibility and convenience. In David's case, polling listeners who send fan emails results in a biased sample because those who send fan emails are more likely to be supporters of his show, leading to an overestimation of positive opinions.

Q: What is response bias, and why is it not the most concerning source of bias in this scenario?

Response bias occurs when individuals do not respond truthfully or are influenced by question wording. In the first scenario, response bias is not the most concerning source of bias because participants are already likely to be fans of David's show. The bias originates from the sampling method.

Q: How does undercoverage differ from volunteer response sampling?

Undercoverage occurs when important demographics are not adequately represented in the sample. Volunteer response sampling, on the other hand, is non-random and occurs when participants choose to respond. Undercoverage is a more clear-cut bias, whereas volunteer response sampling is a type of non-random sampling.

Summary & Key Takeaways

• David wants to know how much his podcast listeners like his show, so he conducts an online poll. However, the poll results may be biased because participants are volunteers and likely to already be fans of David.

• In a different scenario, David decides to poll the next 100 listeners who send him fan emails. This convenience sample is highly skewed towards people who already like his show, leading to an overestimate of the percentage of listeners who love it.

• Response bias occurs when people don't answer truthfully or are influenced by the wording of the question. Undercoverage happens when important demographics are excluded from the sample.