Course contentsShow
← Data Science
Lesson 1497 of 2,14537. A/B TestingFree lesson

Sample Size Formulas for Proportions

Calculate required sample sizes for tests comparing conversion rates or binary outcomes.

Sample Size Formulas for Proportions

You'll learn: How to calculate the exact sample size needed when comparing conversion rates or other binary outcomes in A/B tests.

The Core Idea

When testing proportions (like click-through rates, conversion rates, or sign-up rates), you need a specific formula to determine how many observations you need. Unlike continuous metrics, proportions follow a binomial distribution, which has unique variance properties that affect the calculation.

The standard formula for two-proportion tests is:

n = (Z_α/2 + Z_β)² × [p₁(1-p₁) + p₂(1-p₂)] / (p₁ - p₂)²

Where:

  • n = required sample size per group
  • Z_α/2 = z-score for your significance level (e.g., 1.96 for α=0.05)
  • Z_β = z-score for your desired power (e.g., 0.84 for 80% power)
  • p₁ = baseline proportion (control group conversion rate)
  • p₂ = expected proportion in treatment (p₁ + MDE)
  • (p₁ - p₂) = your minimum detectable effect

Why This Formula?

The variance of a proportion depends on the proportion itself: it's highest at 0.5 and lower near 0 or 1. That's why baseline conversion rate matters—testing a 5% vs 6% conversion requires far more data than testing 45% vs 50%, even though both represent 1 percentage point differences.

Practical Example

If your current conversion rate is 10% (p₁ = 0.10), you want to detect a 2 percentage point lift (p₂ = 0.12), with 80% power and α=0.05:

n = (1.96 + 0.84)² × [0.10×0.90 + 0.12×0.88] / (0.02)²
n ≈ 7.84 × 0.1956 / 0.0004
n ≈ 3,836 per group

Key Takeaway: Sample size formulas for proportions account for the variance structure of binary outcomes—lower baseline rates and smaller effect sizes dramatically increase the data requirements for reliable A/B tests.