Course contentsShow
← Computer Science
Lesson 2815 of 2,87262. Machine Learning FundamentalsFree lesson

Early Stopping

Preventing overfitting by monitoring validation performance and stopping training when it begins to degrade.

Early Stopping

You'll learn how to prevent overfitting by halting training at just the right moment—before your model memorizes noise.

The Core Idea

Early stopping is a simple yet powerful regularization technique: you monitor your model's performance on a validation set during training and stop when that performance starts getting worse, even if training loss is still improving.

Think of it like studying for an exam. At some point, cramming more facts becomes counterproductive—you start memorizing trivia instead of understanding concepts. Early stopping is knowing when to close the book.

How It Works

During training, you track two curves:

  • Training loss: typically decreases continuously
  • Validation loss: decreases, then plateaus, then increases

The moment validation loss stops improving (or starts rising) signals overfitting—your model is learning training-specific noise rather than generalizable patterns. Early stopping says: "Stop here, this was your best model."

Implementation Strategy

  1. Reserve a validation set (separate from test data)
  2. After each training epoch, evaluate on validation set
  3. Track the best validation score seen so far
  4. If validation performance doesn't improve for N epochs (the "patience"), stop training
  5. Restore the model weights from the best validation epoch

Why It Works

Early stopping provides implicit regularization without adding penalty terms. It prevents the optimization process from finding overly complex solutions that fit training noise. You're essentially using validation performance as a signal to halt before your model goes from learning to memorizing.

Key Takeaway: Early stopping is a practical regularization technique that stops training when validation performance degrades, saving the model at its peak generalization—preventing overfitting without explicit parameter penalties.