How Do Bayes Factors Explain Medical Tests?

TL;DR

A positive medical test updates the prior probability of disease rather than directly reporting the probability that a patient is sick. With 1% prevalence, 90% sensitivity, and 91% specificity, only about 1 in 11 positive results represent cancer. Dividing sensitivity by the false positive rate gives a Bayes factor of 10, which supports quick estimates when the prior is small.

Transcript

Some of you may have heard this paradoxical fact about medical tests. It's very commonly used to introduce the topic of Bayes' rule in probability. The paradox is that you could take a test which is highly accurate, in the sense that it gives correct results to a large majority of the people taking it. And yet, under the right circumstances, when a... Read More

Key Insights

• An accurate test is not necessarily highly predictive because its positive predictive value depends on disease prevalence as well as sensitivity and specificity. Even when more than 90% of all patients receive correct results, most positive results can be false positives in a low-prevalence population.
• Positive predictive value is the number of true positives divided by all positive results. In the example, 9 true positives and 89 false positives produce a value of 9 divided by 98, so a positive result indicates only about a 1 in 11 probability of cancer.
• Sensitivity is the proportion of people with the disease who receive a positive result. The example has 90% sensitivity because 9 of 10 women with cancer test positive, while the remaining woman receives a false negative, giving a false negative rate of 10%.
• Specificity is the proportion of people without the disease who receive a negative result. The example has about 91% specificity, while its false positive rate is 9%. These measures describe outcomes among healthy people, not the probability of disease after a positive result.
• Disease prevalence is the prior probability before the test result is known. When prevalence is only 1%, false positives from the much larger healthy group can greatly outnumber true positives, even when the test has sensitivity and specificity above 90%.
• A medical test updates a patient's probability of disease rather than directly determining it. In the example, the prior probability is 1 in 100, and the positive result raises it by almost an order of magnitude to approximately 1 in 11.
• The Bayes factor for a positive result is sensitivity divided by the false positive rate. With 90% sensitivity and a 9% false positive rate, the factor is 10, meaning a positive result is 10 times more likely with cancer than without it.
• The small-prior approximation multiplies the prior probability by the Bayes factor. A prior of 1 in 1,000 therefore becomes approximately 1 in 100 after a positive result. The shortcut cannot handle high priors reliably because it can incorrectly predict probabilities reaching 100% certainty.

Questions & Answers

Q: How can an accurate medical test have low predictive value?

An accurate test can have low positive predictive value when the disease is rare. With 1% prevalence, only 10 of 1,000 women have cancer. At 90% sensitivity, 9 of them test positive. A 9% false positive rate among the other 990 women produces 89 false positives. Consequently, only 9 of 98 positive results represent cancer, approximately 1 in 11.

Q: What is positive predictive value in medical testing?

Positive predictive value, or PPV, is the number of true positive results divided by the total number of positive results. It answers how strongly a positive test predicts that the person actually has the disease. In the example, there are 9 true positives and 89 false positives, so PPV is 9 divided by 98, which is approximately 1 in 11.

Q: What is the difference between sensitivity and specificity?

Sensitivity measures how often a test correctly produces a positive result among people who have the disease. Specificity measures how often it correctly produces a negative result among people who do not have the disease. In the example, sensitivity is 90%, specificity is about 91%, the false negative rate is 10%, and the false positive rate is 9%.

Q: Why does disease prevalence affect a positive test result?

Disease prevalence determines the prior probability that a person has the disease before testing. When prevalence is low, the healthy population is much larger than the diseased population. Even a modest false positive rate applied to that large healthy group can create far more false positives than true positives, making the probability of disease after a positive result surprisingly low.

Q: What is the Bayes factor for a positive medical test?

The Bayes factor for a positive result is calculated by dividing the test sensitivity by its false positive rate. It expresses how much more likely a positive result is among people with the disease than among people without it. With 90% sensitivity and a 9% false positive rate, the Bayes factor is 10, providing a compact measure of the test's updating strength.

Q: How can you quickly estimate probability after a positive test?

For a small prior probability, multiply the prior by the Bayes factor to estimate the updated probability. With a prior of 1 in 100 and a Bayes factor of 10, the estimate is about 1 in 10, slightly above the exact result of roughly 1 in 11. With a prior of 1 in 1,000, the same shortcut gives approximately 1 in 100.

Q: Why does the Bayes factor shortcut fail for high priors?

The shortcut works as an approximation only when the prior probability is small. If the prior is 10% and the Bayes factor is 10, simple multiplication predicts 100% certainty, which cannot be the correct update. In a population of 100, about 9 true positives and about 8 false positives would occur, so positive results would still include many people without cancer.

Q: How do natural frequencies make medical test results clearer?

Natural frequencies replace abstract percentages with counts in a sample population. For 1,000 women with 1% prevalence, the counts are 10 cancer cases and 990 non-cancer cases. Applying the test rates gives 9 true positives and 89 false positives. Comparing these visible groups makes the approximately 1 in 11 positive predictive value much easier to calculate and understand.

Summary & Key Takeaways

• A screening example starts with 1,000 women, of whom 1% have breast cancer. The test produces 9 true positives, 1 false negative, 89 false positives, and 901 true negatives. Therefore, a woman with a positive result has a cancer probability of 9 divided by 98, approximately 1 in 11.

• The apparent paradox comes from confusing test performance with positive predictive value. Sensitivity measures correctness among people with the disease, while specificity measures correctness among those without it. Neither statistic alone answers whether a person with a positive result has the disease, because that conclusion also depends strongly on the disease prevalence.

• A positive result should be viewed as an update to existing chances. The Bayes factor for a positive result is sensitivity divided by the false positive rate. In the example, it equals 10. For a small prior, multiplying by this factor gives a useful mental approximation of the updated probability, though it fails for larger priors.