Post-Hoc Tests and Multiple Comparisons
What you'll learn: How to identify exactly which groups differ after ANOVA finds a significant result.
The Problem ANOVA Leaves Behind
When your one-way ANOVA F-test rejects the null hypothesis, it tells you "at least one group mean differs from the others"—but it doesn't tell you which groups differ. If you have three groups (A, B, C), are A and B different? B and C? All three? That's where post-hoc tests come in.
Why Not Just Run Multiple t-Tests?
The temptation is to compare every pair with a t-test. With 3 groups, that's 3 comparisons; with 5 groups, it's 10 comparisons. But here's the catch: each test at α = 0.05 has a 5% Type I error rate. Multiple tests inflate your overall false-positive risk dramatically—this is the multiple comparisons problem.
Common Post-Hoc Procedures
Tukey's HSD (Honestly Significant Difference)
Tukey's method compares all pairs of group means while controlling the family-wise error rate at your chosen α level. It uses the studentized range distribution (related to the t-distribution) and is ideal when you want to test every possible pair equally.
Bonferroni Correction
The simplest approach: if you're making m comparisons, test each one at significance level α/m. For 10 comparisons at α = 0.05, each individual test uses 0.005. It's conservative (reduces power) but universally applicable and easy to understand.
Key Takeaway: Post-hoc tests like Tukey's HSD and Bonferroni correction let you safely determine which specific group pairs differ significantly after ANOVA, while controlling the inflated risk of false positives that comes with multiple comparisons.