Sample Size Formulas for Continuous Metrics
What you'll learn: How to calculate the number of observations needed when your A/B test measures continuous outcomes like revenue per user or session duration.
The Core Idea
When testing continuous metrics (means) rather than conversion rates (proportions), you need a different sample size formula. Instead of dealing with success/failure outcomes, you're working with measurements that can take any value along a scale—like average order value ($47.23), time on site (3.4 minutes), or clicks per session (8.2).
The key difference: continuous metrics require you to estimate the standard deviation of your metric, not just the baseline rate.
The Formula
For a two-sample t-test comparing means between control and treatment:
n = 2 × (Z_α/2 + Z_β)² × (σ²) / (MDE²)
Where:
- n = sample size per group
- Z_α/2 = critical value for significance level (1.96 for α=0.05)
- Z_β = critical value for power (0.84 for 80% power)
- σ = pooled standard deviation of your metric
- MDE = minimum detectable effect (absolute difference in means)
Practical Example
Say you want to test if a new checkout flow increases average revenue per user. Historical data shows σ = 3 increase (MDE = 3) with 80% power at 5% significance:
n = 2 × (1.96 + 0.84)² × (15²) / (3²)
n = 2 × 7.84 × 225 / 9
n ≈ 392 users per group (784 total)
Key Takeaway: For continuous metrics, sample size depends critically on the variability (standard deviation) of your metric—higher variance requires proportionally more observations to detect the same absolute effect size.