Course contentsShow
← Data Science
Lesson 1499 of 2,14537. A/B TestingFree lesson

Adjusting for Baseline Conversion Rates

Understand how baseline rates affect required sample sizes and plan accordingly for high/low rates.

Adjusting for Baseline Conversion Rates

You'll learn how baseline conversion rates dramatically affect the sample size needed for your A/B test, and why ignoring this can lead to underpowered experiments.

Why Baseline Rates Matter

When calculating sample size for an A/B test involving proportions (like conversion rates), the baseline conversion rate — your current rate before any intervention — has a huge impact on required sample size.

Here's the intuition: detecting the same absolute change (say, +2 percentage points) is much harder when your baseline rate is near 0% or 100% than when it's around 50%.

The Variance Connection

The variance of a proportion follows the formula p(1-p). This variance is maximized when p = 0.5 and minimized at the extremes (0 or 1). Larger variance means you need fewer samples to detect the same effect size — there's more "signal" to work with.

Low baseline rate example: If your current conversion is 2%, improving to 3% (a 50% relative lift!) requires detecting a tiny absolute change. The variance is small (0.02 × 0.98 = 0.0196), so you need enormous sample sizes.

Moderate baseline rate example: If your baseline is 50%, improving to 51% has much higher variance (0.50 × 0.50 = 0.25), making the effect easier to detect with smaller samples.

Practical Planning

When planning your test, always check your baseline rate first. Extreme baselines (below 5% or above 95%) may require 5-10x more samples than moderate rates for the same absolute effect size. Consider using relative effect sizes (e.g., "10% relative improvement") rather than absolute ones when baseline rates are extreme.

Key Takeaway: Baseline conversion rates near extremes (0% or 100%) require dramatically larger sample sizes than moderate rates due to lower variance — always account for your actual baseline when calculating power.