Course contentsShow
← Machine Learning and Deep Learning
Lesson 513 of 3,53812. Model Validation and TuningFree lesson

Successive Halving and Early Stopping

Allocate resources adaptively by terminating poorly performing configurations early.

Successive Halving and Early Stopping

What you'll learn: How to save time and computational resources by intelligently abandoning unpromising hyperparameter configurations before they finish training.


When tuning hyperparameters, you could train every configuration fully—but that wastes resources on clearly bad settings. Successive Halving is a smarter approach: start by training many configurations with a small budget (few iterations, small data subset), then progressively eliminate the worst performers and give more resources only to the promising ones.

How It Works

Imagine you're auditioning 16 singers for a single role. Instead of giving everyone 30 minutes each, you:

  1. Give all 16 singers 2 minutes (small budget)
  2. Keep the top 8, eliminate the rest
  3. Give those 8 singers 5 more minutes
  4. Keep the top 4
  5. Continue halving and increasing budget until one winner emerges

In hyperparameter tuning, your "budget" might be training epochs, dataset size, or compute time. Successive Halving trains many configurations cheaply, then repeatedly halves the surviving candidates while doubling their allocated resources.

Early Stopping Connection

This relates to early stopping (a broader technique): you monitor a metric during training and quit if performance stops improving. Successive Halving applies this principle across configurations, not just within one model's training—it stops investing in configurations that lag behind their peers.

Why This Matters

Instead of grid searching 100 configurations fully (expensive!), Successive Halving might fully train only the top 3-4, having eliminated poor performers early. You get comparable results with a fraction of the computational cost—especially valuable when training deep neural networks or working with large datasets.

Key Takeaway:
Successive Halving allocates compute adaptively—training many configurations with small budgets first, then iteratively eliminating weak performers and investing more resources only in promising candidates, dramatically reducing tuning costs.