Classifier Guidance: Theory
What you'll learn: How to steer diffusion model outputs toward specific classes using gradients from a separate classifier network.
The Core Idea
Classifier guidance enhances class-conditional diffusion by using an external classifier trained to recognize classes in noisy images. During sampling, you calculate the gradient of this classifier's predicted probability for your desired class, then "push" the diffusion process in that direction. Think of it like a GPS constantly nudging your car toward the destination—the classifier continuously guides the noisy image toward the target class as it denoises.
How It Works
At each sampling step with noise level t, you have:
- The normal predicted noise from your diffusion model
- The gradient of
log p(y|x_t)from your classifier (whereyis the desired class andx_tis the current noisy image)
You combine these using a guidance scale w:
- Stronger guidance (larger
w) = more class-conforming but potentially less diverse samples - Weaker guidance = more diverse but less faithful to the target class
The classifier must be trained on noisy images at various timesteps, not just clean data, because it needs to classify partially-denoised samples during generation.
The Mathematical Intuition
The classifier's gradient points in the direction that increases the probability of the target class. By adding a scaled version of this gradient to the diffusion model's prediction, you're essentially saying: "Denoise this image, but also make it look more like class X."
Key Takeaway: Classifier guidance uses gradients from a noise-aware classifier to steer diffusion sampling toward desired classes, trading off between sample quality (class fidelity) and diversity through a guidance scale parameter.