Course contentsShow
← Machine Learning and Deep Learning
Lesson 1475 of 3,53833. Generative Adversarial NetworksFree lesson

The GAN Training Loop

Step-by-step process of alternating discriminator and generator updates during training.

The GAN Training Loop

What you'll learn: How GANs train by alternating between updating the discriminator and generator in each training iteration.

The Alternating Update Process

Unlike typical neural networks with a single loss to minimize, GANs require carefully coordinating two competing networks. The training loop alternates between strengthening the discriminator and fooling it with the generator.

Think of it like a counterfeiter (generator) and detective (discriminator) training together. The detective studies real bills and fakes, getting better at spotting counterfeits. Then the counterfeiter creates new fakes based on what the detective just learned. This back-and-forth continues until the fakes become indistinguishable from real bills.

Step-by-Step Training

Phase 1: Train the Discriminator

  • Sample real data from your dataset
  • Generate fake data using the current generator
  • Forward pass both through the discriminator
  • Calculate discriminator loss (how well it distinguishes real from fake)
  • Backpropagate and update only discriminator weights
  • The generator stays frozen during this step

Phase 2: Train the Generator

  • Generate new fake samples
  • Forward pass through discriminator (now frozen)
  • Calculate generator loss (how well it fooled the discriminator)
  • Backpropagate through discriminator into generator
  • Update only generator weights

These two phases repeat for thousands of iterations. The discriminator learns to classify better, forcing the generator to produce more realistic samples, which in turn makes the discriminator's job harder—driving both toward the Nash equilibrium.

The key is maintaining balance: if one network becomes too strong too quickly, training can collapse.

Key Takeaway: The GAN training loop alternates between discriminator updates (learning to detect fakes) and generator updates (learning to fool the discriminator), creating an adversarial arms race that improves both networks.