Course contentsShow
← Machine Learning and Deep Learning
Lesson 1557 of 3,53834. Diffusion ModelsFree lesson

Annealed Langevin Dynamics

Understand multi-scale sampling where noise levels are gradually reduced to refine samples from coarse to fine.

Annealed Langevin Dynamics

You'll learn how to refine samples by gradually reducing noise levels, moving from coarse to fine-grained generation.

The Core Idea

In previous lessons, you learned about Langevin dynamics for sampling and noise conditional score networks. Annealed Langevin Dynamics combines these ideas by using multiple noise levels in sequence, starting high and gradually decreasing.

Think of it like sculpting: you start with rough, bold strokes to establish the overall shape, then progressively use finer tools to add detail. Similarly, annealed Langevin dynamics begins sampling with high noise (capturing broad structure) and progressively reduces noise to refine details.

How It Works

Instead of sampling at a single noise level, you:

  1. Start with high noise — The score network guides sampling in a very noisy regime where large-scale structure emerges
  2. Gradually decrease noise — Step through a schedule of decreasing noise levels (σ₁ > σ₂ > ... > σₙ)
  3. Run Langevin steps at each level — At each noise level, perform several Langevin dynamics updates before moving to the next (lower) noise level
  4. End with minimal noise — The final steps operate near the clean data distribution

This multi-scale approach helps avoid getting stuck in local modes and produces higher-quality samples. The annealing schedule acts like a "curriculum" for sampling, solving easier (noisier) problems first before tackling fine details.

Why It Matters

Annealed Langevin dynamics bridges score matching with practical high-quality generation. It's the foundation for understanding how modern diffusion models efficiently generate samples through a hierarchy of noise levels.

Key Takeaway: Annealed Langevin dynamics refines samples from coarse to fine by gradually reducing noise levels, enabling multi-scale generation that captures both global structure and local details.