Course contentsShow
← Data Science
Lesson 1511 of 2,14537. A/B TestingFree lesson

Sequential Probability Ratio Test (SPRT)

How SPRT uses likelihood ratios to make stopping decisions with predefined Type I and Type II error rates.

Sequential Probability Ratio Test (SPRT)

You'll learn how SPRT uses likelihood ratios to decide when to stop an A/B test early while controlling both false positive and false negative rates.

The Core Idea

Unlike fixed-horizon tests that wait for a predetermined sample size, SPRT examines data as it arrives and uses a likelihood ratio to make stopping decisions. At each step, SPRT asks: "Based on the data so far, is the evidence strong enough to make a decision, or should I keep collecting?"

The test compares two hypotheses:

  • H₀: No difference between control and treatment
  • H₁: A specific effect size exists

For each observation, SPRT calculates a likelihood ratio — how much more likely the data is under H₁ compared to H₀. This ratio accumulates evidence over time.

How It Controls Error Rates

SPRT sets two thresholds based on your desired error rates:

  • Upper threshold (B): Based on acceptable Type I error (α, false positive rate)
  • Lower threshold (A): Based on acceptable Type II error (β, false negative rate)

The decision rule is simple:

  • If likelihood ratio ≥ B → Stop and reject H₀ (declare a winner)
  • If likelihood ratio ≤ A → Stop and accept H₀ (no significant difference)
  • If A < ratio < B → Continue collecting data

Think of it like a balance scale: you keep adding evidence (data) until the scale tips decisively one way or the other, with the thresholds defining "decisive."

Why This Matters for A/B Tests

SPRT lets you stop early when results are clear, reducing sample size requirements on average while guaranteeing your Type I and Type II error rates stay at or below your predefined levels (α and β).

Key Takeaway: SPRT uses accumulated likelihood ratios between two thresholds to make stopping decisions, guaranteeing specified Type I and Type II error rates while potentially reducing test duration.