Interpreting Rank-Based Tests: Medians vs Distributions
What You'll Learn
You'll discover what non-parametric rank-based tests actually tell you—and why it's more than just comparing medians.
The Common Misconception
When you run a Mann-Whitney U test or Wilcoxon test, it's tempting to think: "This test compares medians between groups." While these tests can detect median differences, that's not their full story.
What they actually test: Whether two groups have identical distributions.
Understanding the Full Picture
Think of distributions like fingerprints—they have location (where they're centered), spread (how wide they are), and shape (symmetric, skewed, etc.). Rank-based tests are sensitive to all these features, not just the median.
When Medians Align
Imagine:
- Group A: Values tightly clustered around 50 (median = 50)
- Group B: Values spread widely from 20 to 80 (median = 50)
Both have the same median, but a Mann-Whitney U test might still reject the null hypothesis because the distributions differ in spread. The ranks will look different even though the centers match.
The Key Question
These tests ask: "If I randomly pick one value from each group, is one equally likely to be larger than the other?" This question captures the entire distributional difference, not just central tendency.
When Medians Tell the Story
If you can assume both groups have the same shape and spread (just shifted locations), then a significant rank test reliably indicates different medians. But without that assumption, you're testing something broader: distributional equality.
Key Takeaway: Non-parametric rank-based tests detect differences in entire distributions (location, spread, and shape), not just medians. Only interpret them as "median tests" when you're confident the groups have similar distributional shapes.