The Logic of Hypothesis Testing
What You'll Learn
You'll understand how hypothesis testing uses proof by contradiction to make statistical decisions—rejecting or failing to reject the null hypothesis based on evidence.
The Core Logic: Proof by Contradiction
Hypothesis testing borrows a classic mathematical strategy: proof by contradiction. Here's how it works:
- Assume the opposite of what you suspect is true (this is the null hypothesis, H₀)
- Collect evidence and see what happens under that assumption
- If the evidence is extremely unlikely under your assumption, reject that assumption
- If the evidence is plausible, you don't have enough proof to reject it
An Everyday Analogy
Imagine you suspect a coin is unfair (favors heads). You can't prove it's unfair directly, so you:
- Assume it's fair (H₀: the coin is fair)
- Flip it 100 times and get 95 heads
- Ask: "If the coin were fair, how likely is 95 heads?"
- Answer: Virtually impossible (extremely small p-value)
- Conclusion: Reject the assumption of fairness
If you had gotten 52 heads instead, that's perfectly consistent with a fair coin, so you'd fail to reject H₀—not because you proved it's fair, but because you lack compelling evidence against it.
The Decision Framework
- Reject H₀: The data is so extreme under H₀ that it's more reasonable to believe H₀ is false
- Fail to reject H₀: The data is consistent with H₀; we have insufficient evidence to conclude otherwise
Notice we never "accept" or "prove" H₀—we simply lack evidence strong enough to reject it. This asymmetry reflects the proof-by-contradiction structure.
Key Takeaway: Hypothesis testing uses proof by contradiction: assume the null hypothesis is true, then reject it only if your observed data would be extremely unlikely under that assumption.