Comparison to VAEs and GANs
What you'll learn: How diffusion models differ fundamentally from VAEs and GANs in their training philosophy and sampling process.
The Three Families Side-by-Side
Now that you've learned about VAEs, GANs, and diffusion models separately, let's see what makes each unique.
VAEs: Explicit Likelihood with Approximate Inference
VAEs learn an explicit probabilistic mapping between data and latent space. They train with a single objective (ELBO) that balances reconstruction and regularization. Sampling is fast—one forward pass through the decoder—but generated samples can be blurry due to the reconstruction loss averaging pixel values.
GANs: Adversarial Game Without Explicit Likelihood
GANs play a minimax game between generator and discriminator. They don't model likelihood explicitly, which can produce sharp, high-quality images. However, training is notoriously unstable (mode collapse, vanishing gradients) and requires careful balancing of two networks. Sampling is also fast: one forward pass.
Diffusion Models: Iterative Refinement with Stable Training
Diffusion models learn to reverse a gradual noising process. Training is remarkably stable—it's essentially learning to denoise at different noise levels, a straightforward supervised task. No adversarial dynamics, no posterior collapse. The trade-off? Sampling requires many iterative steps (often 50-1000), making generation much slower than VAEs or GANs.
Think of it this way: VAEs are like sketching with soft pencils (blurry but reliable), GANs are like racing sports cars (fast and sharp but hard to control), and diffusion models are like careful sculptors (slow but consistently excellent results).
Key Takeaway: Diffusion models sacrifice sampling speed for stable training and high sample quality, avoiding the blurriness of VAEs and the training instability of GANs through iterative denoising.