Course contentsShow
← Data Science
Lesson 1492 of 2,14537. A/B TestingFree lesson

Rerandomization and Practical Implementation

Explore advanced techniques like rerandomization and practical tools for implementing randomization schemes.

Rerandomization and Practical Implementation

What you'll learn: How rerandomization improves balance in A/B tests and how to implement randomization schemes in practice.

The Rerandomization Concept

Even with stratified randomization, you can get unlucky—sometimes random assignment produces groups that look different on important covariates just by chance. Rerandomization is a technique where you check covariate balance before starting your experiment, and if balance is poor, you rerandomize until you get acceptable balance.

Think of it like shuffling a deck of cards: if you're dealing hands for a tournament and one player accidentally gets four aces, you reshuffle. Rerandomization follows the same logic—reject imbalanced randomizations and try again.

How It Works

  1. Define a balance criterion (e.g., maximum allowed Mahalanobis distance between groups on key covariates)
  2. Generate a randomization using your chosen method
  3. Check covariate balance against your threshold
  4. Accept or reject: if balance is good, proceed; if not, generate a new randomization
  5. Repeat until balance meets your standard

This creates a restricted randomization distribution—you're only using assignments that pass your balance check.

Practical Implementation Tips

  • Automate the process: write scripts that loop through randomizations and check balance
  • Set thresholds in advance: document your acceptance criteria before seeing data
  • Track randomization seed: always save the random seed used for reproducibility
  • Balance inference: remember that rerandomization changes your p-values slightly (though often negligibly in practice)
  • Use blocking first: rerandomization works best after applying stratification—it fine-tunes balance within strata

Key Takeaway: Rerandomization improves covariate balance by rejecting poorly balanced assignments before your experiment begins, reducing variance and increasing power while maintaining valid causal inference when done correctly.