Course contentsShow
← Data Science
Lesson 397 of 2,14510. Hypothesis Tests for MeansFree lesson

Power and Efficiency of Non-Parametric Tests

Compare the statistical power of rank tests to t-tests and understand when each is more efficient.

Power and Efficiency of Non-Parametric Tests

You'll learn: How rank-based tests compare to t-tests in detecting true effects, and when each approach gives you the best bang for your buck.

The Core Trade-off

When you choose between a t-test and a non-parametric alternative (like the Mann-Whitney U or Wilcoxon Signed-Rank), you're making a trade-off between power and robustness.

Statistical power is your test's ability to detect a real effect when it exists. If your data truly meet the normality assumptions, t-tests are more efficient—they squeeze more information from the same sample size. Think of efficiency like fuel economy: a t-test on normal data travels further per data point.

However, when assumptions fail (non-normal data, outliers, skewness), non-parametric tests often have better power than t-tests because they don't get distorted by assumption violations.

The Efficiency Numbers

Under ideal conditions (perfectly normal data), non-parametric tests have approximately 95% efficiency compared to t-tests. This means you'd need about 5% more data with a rank test to achieve the same power as a t-test.

That's a small price to pay for insurance! When data are heavily skewed or contaminated with outliers, rank tests can be far more powerful than t-tests—sometimes needing half the sample size to detect the same effect.

When Each Shines

  • Use t-tests when: data are approximately normal, moderate sample sizes, you want maximum power from clean data
  • Use rank tests when: data are skewed, outliers present, small samples where you can't verify normality, or you care about distribution shifts beyond just means

Key Takeaway: Non-parametric tests sacrifice ~5% efficiency on perfect data but gain substantial power when assumptions fail—making them the safer default choice when you're uncertain about your data's distribution.