Course contentsShow
← Machine Learning and Deep Learning
Lesson 1502 of 3,53833. Generative Adversarial NetworksFree lesson

Measuring Training Stability

Monitoring discriminator and generator losses, along with other signals to detect training instability early.

Measuring Training Stability

What you'll learn: How to monitor key signals during GAN training to catch instability before it derails your model.


Why Stability Monitoring Matters

Training GANs is like watching two opponents in a delicate dance—when one stumbles, both fall. Unlike standard neural networks where loss steadily decreases, GAN training involves competing objectives that can spiral out of control. You need early warning systems to catch problems before they become catastrophic.

Core Signals to Track

Loss Trajectories

Monitor both discriminator loss and generator loss over time. Healthy training shows:

  • Oscillating but bounded losses: They should fluctuate but stay within a reasonable range
  • No divergence: Losses shouldn't shoot toward infinity or collapse to zero
  • Rough balance: Neither network should completely dominate (though exact equality isn't required)

Warning Signs

Watch for these red flags:

  • Generator loss increasing monotonically: The discriminator is winning too easily
  • Discriminator loss approaching zero: It's becoming too confident, starving the generator of gradients
  • Wild oscillations: Losses swinging dramatically suggest unstable dynamics
  • Flat generator loss with high discriminator loss: Often indicates mode collapse

Additional Stability Metrics

Beyond raw losses, track:

  • Gradient norms: Sudden spikes or vanishing values signal trouble
  • Discriminator confidence: Average output on real vs. fake samples
  • Generated sample diversity: Visual inspection or automated metrics

Practical Monitoring

Log these metrics every few iterations. Plot them during training. If you see divergence patterns emerging, you can intervene with learning rate adjustments, architecture tweaks, or regularization before wasting hours on a doomed run.

Key Takeaway: Stable GAN training requires actively monitoring discriminator and generator losses alongside gradient norms and output diversity—catching instability early lets you intervene before training completely fails.