Sample Size Formulas for Proportions
You'll learn: How to calculate the exact sample size needed when comparing conversion rates or other binary outcomes in A/B tests.
The Core Idea
When testing proportions (like click-through rates, conversion rates, or sign-up rates), you need a specific formula to determine how many observations you need. Unlike continuous metrics, proportions follow a binomial distribution, which has unique variance properties that affect the calculation.
The standard formula for two-proportion tests is:
n = (Z_α/2 + Z_β)² × [p₁(1-p₁) + p₂(1-p₂)] / (p₁ - p₂)²
Where:
- n = required sample size per group
- Z_α/2 = z-score for your significance level (e.g., 1.96 for α=0.05)
- Z_β = z-score for your desired power (e.g., 0.84 for 80% power)
- p₁ = baseline proportion (control group conversion rate)
- p₂ = expected proportion in treatment (p₁ + MDE)
- (p₁ - p₂) = your minimum detectable effect
Why This Formula?
The variance of a proportion depends on the proportion itself: it's highest at 0.5 and lower near 0 or 1. That's why baseline conversion rate matters—testing a 5% vs 6% conversion requires far more data than testing 45% vs 50%, even though both represent 1 percentage point differences.
Practical Example
If your current conversion rate is 10% (p₁ = 0.10), you want to detect a 2 percentage point lift (p₂ = 0.12), with 80% power and α=0.05:
n = (1.96 + 0.84)² × [0.10×0.90 + 0.12×0.88] / (0.02)²
n ≈ 7.84 × 0.1956 / 0.0004
n ≈ 3,836 per group
Key Takeaway: Sample size formulas for proportions account for the variance structure of binary outcomes—lower baseline rates and smaller effect sizes dramatically increase the data requirements for reliable A/B tests.