Label Smoothing for GANs
You'll learn how to prevent your discriminator from becoming overconfident by softening the labels it trains on, improving gradient flow and training stability.
What Is Label Smoothing?
In standard GAN training, you typically use hard labels: real images get label 1.0 and fake images get label 0.0. The discriminator learns to classify images as perfectly real or fake. But this creates a problem—when the discriminator becomes too confident, it outputs values very close to 0 or 1, causing gradients to vanish. The generator then receives weak learning signals and struggles to improve.
Label smoothing softens these target values. Instead of 1.0 for real images, you might use 0.9. This is called one-sided label smoothing because you typically only smooth the "real" labels, not the fake ones.
Why One-Sided?
Smoothing only the real labels (e.g., using 0.9 instead of 1.0) tells the discriminator: "Don't be 100% certain these are real—leave some room for doubt." This prevents overconfidence while still maintaining a clear distinction between real and fake.
You generally don't smooth fake labels away from 0.0 because that would encourage the discriminator to incorrectly classify some fake images as real, which defeats its purpose.
The Effect on Training
By preventing the discriminator from reaching extreme confidence, label smoothing:
- Keeps gradients flowing back to the generator
- Reduces the likelihood of mode collapse
- Improves training stability
- Helps both networks continue learning throughout training
Think of it like grading a student: instead of only giving 100% or 0%, you give 90% even for excellent work, encouraging continuous improvement rather than complacency.
Key Takeaway: One-sided label smoothing (e.g., using 0.9 instead of 1.0 for real labels) prevents discriminator overconfidence, maintains healthier gradients, and stabilizes GAN training.