Sequential Probability Ratio Test (SPRT)
You'll learn how SPRT uses likelihood ratios to decide when to stop an A/B test early while controlling both false positive and false negative rates.
The Core Idea
Unlike fixed-horizon tests that wait for a predetermined sample size, SPRT examines data as it arrives and uses a likelihood ratio to make stopping decisions. At each step, SPRT asks: "Based on the data so far, is the evidence strong enough to make a decision, or should I keep collecting?"
The test compares two hypotheses:
- H₀: No difference between control and treatment
- H₁: A specific effect size exists
For each observation, SPRT calculates a likelihood ratio — how much more likely the data is under H₁ compared to H₀. This ratio accumulates evidence over time.
How It Controls Error Rates
SPRT sets two thresholds based on your desired error rates:
- Upper threshold (B): Based on acceptable Type I error (α, false positive rate)
- Lower threshold (A): Based on acceptable Type II error (β, false negative rate)
The decision rule is simple:
- If likelihood ratio ≥ B → Stop and reject H₀ (declare a winner)
- If likelihood ratio ≤ A → Stop and accept H₀ (no significant difference)
- If A < ratio < B → Continue collecting data
Think of it like a balance scale: you keep adding evidence (data) until the scale tips decisively one way or the other, with the thresholds defining "decisive."
Why This Matters for A/B Tests
SPRT lets you stop early when results are clear, reducing sample size requirements on average while guaranteeing your Type I and Type II error rates stay at or below your predefined levels (α and β).
Key Takeaway: SPRT uses accumulated likelihood ratios between two thresholds to make stopping decisions, guaranteeing specified Type I and Type II error rates while potentially reducing test duration.