Course contentsShow
← Machine Learning and Deep Learning
Lesson 526 of 3,53812. Model Validation and TuningFree lesson

Diagnosing Convergence Issues

Using loss curves to detect non-convergence, oscillation, and learning rate problems.

Diagnosing Convergence Issues

What you'll learn: How to use loss curves to spot when your model isn't learning properly, why it might be bouncing around, and how to identify learning rate problems.

The Core Idea

A loss curve shows how your model's error changes during training. When things go well, the loss steadily decreases and eventually flattens out (convergence). But sometimes training goes wrong—the model never settles down, bounces wildly, or gets stuck. Reading these curves is like checking a patient's vital signs: irregular patterns reveal specific problems.

Three Key Diagnosis Patterns

Non-Convergence (Plateau Too Early)
If your loss stops improving while still high, the model hasn't reached a good solution. Think of a hiker who stops halfway up the mountain. This often means your model is too simple, your learning rate is too small (taking baby steps), or you stopped training too soon.

Oscillation (Bouncing Up and Down)
When the loss jumps erratically or zigzags wildly instead of smoothly decreasing, your learning rate is likely too high. Imagine trying to descend a hill by taking giant leaps—you'll overshoot the valley and bounce from side to side, never settling at the bottom.

Learning Rate Problems
The "just right" learning rate creates a smooth downward curve that gradually flattens. Too low: painfully slow progress. Too high: chaotic jumping or even increasing loss (climbing instead of descending).

What to Look For

  • Smooth downward trend = healthy training
  • Flat line at high loss = non-convergence
  • Wild jumps = learning rate too high
  • Barely moving = learning rate too low

Key Takeaway: Loss curves are your diagnostic tool—a smooth downward trend signals healthy convergence, while plateaus, oscillations, or explosions reveal specific learning rate and optimization problems you need to fix.