Reverse-Time SDE for Generation
What you'll learn: How to mathematically reverse the forward diffusion process using an SDE that runs backward in time to generate samples.
The Core Idea
In the previous lesson, you saw how forward diffusion can be described as a Stochastic Differential Equation (SDE) that gradually adds noise. Now comes the magic: we can run that process backward in time to generate samples from pure noise!
The reverse-time SDE looks like this:
dx = [f(x,t) - g(t)² ∇ₓ log p_t(x)] dt + g(t) dw̄
Where:
- f(x,t) and g(t) are from your forward SDE
- ∇ₓ log p_t(x) is the score function (gradient of log-probability)
- dw̄ is a backward-time Brownian motion
- Time flows from T → 0 (backward!)
Why This Works
Think of it like filming a movie of ink dispersing in water, then playing it in reverse. The reverse process needs two things:
-
Drift correction: The term
-g(t)² ∇ₓ log p_t(x)acts like a "smart guide" that steers random noise back toward realistic data. The score function tells us which direction increases probability. -
Stochasticity: The
g(t) dw̄term keeps the process random, ensuring diverse samples.
The Generation Recipe
Start with pure noise x_T ~ N(0,I) at time T, then simulate the reverse SDE backward to time 0. The score function ∇ₓ log p_t(x) is what your neural network learns during training—it's the "compass" pointing toward data.
Key Takeaway: The reverse-time SDE generates samples by running diffusion backward using learned scores as a drift correction that guides noise back into realistic data.