False Positives and Base Rate Fallacy
What You'll Learn
You'll understand why surprisingly accurate tests can still produce mostly false alarms when screening for rare conditions, and how to avoid a common logical trap.
The Core Problem
Imagine a medical test that's 99% accurate for detecting a disease that affects only 1 in 1,000 people. If you test positive, your intuition might scream "99% chance I'm sick!" But surprisingly, you're probably fine.
Here's why: When testing 1,000 people:
- 1 person truly has the disease, and the test catches them (99% of 1 ≈ 1 person)
- 999 people are healthy, but the test falsely flags about 10 of them as positive (1% false positive rate × 999)
So of the ~11 positive results, only 1 is a true positive. Your actual probability of having the disease given a positive test is only about 1/11 ≈ 9%, not 99%!
The Base Rate Fallacy
The base rate fallacy occurs when we ignore the prevalence (base rate) of a condition in the population. People focus on the test's accuracy while forgetting how rare the condition is.
Using Bayes' Theorem from our previous lesson:
P(Disease | Positive) = P(Positive | Disease) × P(Disease) / P(Positive)
The tiny P(Disease) = 0.001 dominates the calculation, even though P(Positive | Disease) = 0.99 looks impressive.
Why This Matters
This applies beyond medicine: spam filters, security screening, fraud detection—anywhere we test for rare events. A "highly accurate" system can still produce mostly false positives when the base rate is low.
Key Takeaway: When evaluating test results for rare conditions, always account for the base rate—accuracy alone is misleading, and most positive results may be false alarms even with excellent tests.