Course contentsShow
← Machine Learning and Deep Learning
Lesson 522 of 3,53812. Model Validation and TuningFree lesson

High Variance Diagnosis

Identifying large gaps between training and validation error that signal overfitting.

High Variance Diagnosis

What you'll learn: How to spot overfitting by identifying when your model performs significantly better on training data than validation data.

The Problem: When Your Model Memorizes Instead of Learning

High variance means your model has learned the training data too well—it's memorized noise and specific patterns that don't generalize to new data. It's like a student who memorizes practice exam answers word-for-word but can't solve slightly different problems on the real test.

Spotting High Variance in Learning Curves

From your previous lesson on learning curves, you now know how to diagnose high variance:

The Telltale Gap

Large gap between training and validation error is the smoking gun. When you see:

  • Training error: very low (model fits training data almost perfectly)
  • Validation error: much higher (model struggles with new data)
  • The gap between them is wide and persistent

...you're witnessing high variance. The model isn't generalizing—it's overfitting.

Visual Signature

On a learning curve plot:

  • The training curve drops quickly and stays very low
  • The validation curve remains high above it
  • Adding more training examples narrows the gap slightly, but it remains significant

Why It Matters

High variance wastes your model's capacity on memorizing irrelevant details. A simpler model or one with regularization would likely perform better on real-world data.

Key Takeaway: High variance is diagnosed when training error is much lower than validation error—your model has overfit the training data and fails to generalize. The solution involves regularization, simpler models, or gathering more training data.