Calculating Power for Simple Tests
You'll learn: How to compute the statistical power of z-tests and t-tests when you have specific alternative hypotheses and known parameters.
What Is Power Calculation?
Remember that statistical power is the probability of correctly rejecting the null hypothesis when a specific alternative hypothesis is true. Calculating power lets you quantify how likely your test is to detect a real effect of a given size. This requires specifying not just the null hypothesis, but an exact alternative value for the parameter you're testing.
The General Approach
To calculate power for a z-test or t-test:
- Specify everything: significance level α, sample size n, population standard deviation (or estimate), and the specific alternative parameter value μ₁ you want to detect
- Find the critical value(s) under the null hypothesis that define your rejection region
- Calculate the test statistic's distribution under the alternative hypothesis μ₁
- Compute the probability that this alternative distribution falls in the rejection region
Example Framework
For a one-sample z-test testing H₀: μ = μ₀ against H₁: μ = μ₁ (where μ₁ > μ₀), with α = 0.05:
- The critical value is z_crit = 1.645 (one-tailed)
- Under H₀, reject if the test statistic exceeds 1.645
- Under H₁, the test statistic follows a normal distribution centered at (μ₁ - μ₀)/(σ/√n)
- Power = P(Z > 1.645 - (μ₁ - μ₀)/(σ/√n)) where Z ~ N(0,1)
For t-tests, replace z-values with t-values and use the non-central t-distribution, though for moderate sample sizes the calculations are similar.
Key Takeaway: Power calculations require specifying a concrete alternative hypothesis and computing how likely the test statistic under that alternative falls in the null's rejection region—turning the abstract concept of power into a concrete probability.