Course contentsShow
← Data Science
Lesson 1498 of 2,14537. A/B TestingFree lesson

Sample Size Formulas for Continuous Metrics

Determine sample sizes needed when testing means like revenue per user or session duration.

Sample Size Formulas for Continuous Metrics

What you'll learn: How to calculate the number of observations needed when your A/B test measures continuous outcomes like revenue per user or session duration.

The Core Idea

When testing continuous metrics (means) rather than conversion rates (proportions), you need a different sample size formula. Instead of dealing with success/failure outcomes, you're working with measurements that can take any value along a scale—like average order value ($47.23), time on site (3.4 minutes), or clicks per session (8.2).

The key difference: continuous metrics require you to estimate the standard deviation of your metric, not just the baseline rate.

The Formula

For a two-sample t-test comparing means between control and treatment:

n = 2 × (Z_α/2 + Z_β)² × (σ²) / (MDE²)

Where:

  • n = sample size per group
  • Z_α/2 = critical value for significance level (1.96 for α=0.05)
  • Z_β = critical value for power (0.84 for 80% power)
  • σ = pooled standard deviation of your metric
  • MDE = minimum detectable effect (absolute difference in means)

Practical Example

Say you want to test if a new checkout flow increases average revenue per user. Historical data shows σ = 15.Youwanttodetecta15. You want to detect a 3 increase (MDE = 3) with 80% power at 5% significance:

n = 2 × (1.96 + 0.84)² × (15²) / (3²)
n = 2 × 7.84 × 225 / 9
n ≈ 392 users per group (784 total)

Key Takeaway: For continuous metrics, sample size depends critically on the variability (standard deviation) of your metric—higher variance requires proportionally more observations to detect the same absolute effect size.