Progressive Growing Strategy
What you'll learn: How training GANs by gradually increasing image resolution can stabilize learning and produce higher-quality outputs.
The Core Idea
Imagine trying to paint a massive, detailed mural on your first attempt—you'd likely make mistakes and struggle with consistency. Instead, what if you started by sketching a small, low-resolution version, perfecting the composition, then gradually scaling up while adding finer details?
Progressive growing applies this exact philosophy to GAN training. Rather than immediately training a generator and discriminator on high-resolution images (like 1024×1024), you start with very small images (like 4×4 or 8×8). Once training stabilizes at this resolution, you progressively double the resolution (16×16, then 32×32, 64×64, and so on), adding new layers to both networks as you scale up.
Why It Works
High-resolution GAN training is notoriously unstable—the discriminator can easily overwhelm the generator, leading to mode collapse or vanishing gradients. By starting small:
- Early stability: Low-resolution images are easier to learn, establishing a solid foundation
- Smooth transitions: Each resolution step inherits weights from the previous stage
- Better gradient flow: The generator receives meaningful feedback throughout training
- Faster convergence: Lower resolutions train quickly, and higher resolutions only refine details
Think of it as learning to walk before you run. The generator masters basic structure and composition at low resolution, then progressively learns to add texture, fine details, and subtle variations.
This strategy was popularized by ProGAN (Progressive Growing of GANs) and directly addresses the training instability and vanishing gradient problems you've already encountered.
Key Takeaway: Progressive growing stabilizes GAN training by starting with low-resolution images and gradually increasing resolution, allowing the networks to learn structure before details.