What is Effect Size?
You'll learn: Why finding a statistically significant result doesn't always mean you've found something important in the real world.
The Problem with P-Values Alone
You've learned that hypothesis tests tell you whether an observed difference is statistically significant — unlikely to have occurred by chance. But statistical significance doesn't automatically mean practical importance.
Imagine testing a new blood pressure medication on 10,000 patients. You find it lowers blood pressure by an average of 0.5 mmHg, with a p-value of 0.001. That's highly significant! But is a 0.5 mmHg reduction meaningful to doctors or patients? Probably not.
What Effect Size Measures
Effect size quantifies the magnitude of the difference or relationship you've detected. It answers: "How big is this effect in practical terms?"
While a p-value tells you if something happened, effect size tells you how much it happened.
Why Effect Size Matters
With large sample sizes, even tiny, trivial differences become statistically significant. Conversely, important real-world effects might not reach significance with small samples. Effect size gives you context independent of sample size.
Analogy: A p-value is like a smoke detector — it tells you there's smoke. Effect size tells you whether it's a candle or a house fire.
Common Effect Size Measures
Different tests have different effect size metrics:
- Cohen's d: for t-tests (difference between means in standard deviation units)
- Correlation coefficient (r): for relationships between variables
- Odds ratio: for proportions
You'll learn to calculate specific effect sizes as you work with each test type.
Key Takeaway: Effect size measures the practical magnitude of your findings, complementing statistical significance to help you determine whether results matter in the real world.