Annealed Langevin Dynamics
You'll learn how to refine samples by gradually reducing noise levels, moving from coarse to fine-grained generation.
The Core Idea
In previous lessons, you learned about Langevin dynamics for sampling and noise conditional score networks. Annealed Langevin Dynamics combines these ideas by using multiple noise levels in sequence, starting high and gradually decreasing.
Think of it like sculpting: you start with rough, bold strokes to establish the overall shape, then progressively use finer tools to add detail. Similarly, annealed Langevin dynamics begins sampling with high noise (capturing broad structure) and progressively reduces noise to refine details.
How It Works
Instead of sampling at a single noise level, you:
- Start with high noise — The score network guides sampling in a very noisy regime where large-scale structure emerges
- Gradually decrease noise — Step through a schedule of decreasing noise levels (σ₁ > σ₂ > ... > σₙ)
- Run Langevin steps at each level — At each noise level, perform several Langevin dynamics updates before moving to the next (lower) noise level
- End with minimal noise — The final steps operate near the clean data distribution
This multi-scale approach helps avoid getting stuck in local modes and produces higher-quality samples. The annealing schedule acts like a "curriculum" for sampling, solving easier (noisier) problems first before tackling fine details.
Why It Matters
Annealed Langevin dynamics bridges score matching with practical high-quality generation. It's the foundation for understanding how modern diffusion models efficiently generate samples through a hierarchy of noise levels.
Key Takeaway: Annealed Langevin dynamics refines samples from coarse to fine by gradually reducing noise levels, enabling multi-scale generation that captures both global structure and local details.