Course contentsShow
← Data Science
Lesson 1490 of 2,14537. A/B TestingFree lesson

Choosing Stratification Variables

Understand which variables to stratify on based on their relationship to outcomes and sample size considerations.

Choosing Stratification Variables

You'll learn how to select the right variables for stratification to maximize balance and precision in your A/B test.

Stratification in A/B testing means dividing your sample into homogeneous groups before randomization. But which variables should you stratify on? Not every variable is worth the effort—you need to be strategic.

The Two Core Selection Criteria

1. Strong Relationship to the Outcome

Choose variables that are predictive of your success metric. If a variable strongly correlates with conversion rate, revenue, or whatever you're measuring, stratifying on it will reduce variance and improve precision.

Think of it this way: If you're testing a checkout flow and know that mobile vs. desktop users behave very differently, stratifying by device type ensures both groups get equal representation in control and treatment. This prevents a device imbalance from masking or exaggerating your treatment effect.

2. Practical Sample Size Considerations

Each stratification variable multiplies the number of strata. With two levels per variable (e.g., mobile/desktop, new/returning), two variables create 2×2 = 4 strata. Three variables? Eight strata.

The risk: Too many strata with too few observations each can actually hurt balance. If you have 1,000 users and 16 strata, you're randomizing only ~62 users per stratum—small enough that random chance can create imbalances.

What NOT to Stratify On

  • Variables unrelated to your outcome (adds complexity without benefit)
  • Variables with too many levels that fragment your sample excessively
  • Post-treatment variables (these are influenced by your treatment!)

Key Takeaway: Stratify on pre-treatment variables that strongly predict your outcome metric and keep the number of strata manageable relative to your sample size—typically aim for at least 30-50 units per stratum.