Course contentsShow
← Machine Learning and Deep Learning
Lesson 1560 of 3,53834. Diffusion ModelsFree lesson

Reverse-Time SDE for Generation

Derive the reverse-time SDE that generates samples by reversing the forward noising process using learned scores.

Reverse-Time SDE for Generation

What you'll learn: How to mathematically reverse the forward diffusion process using an SDE that runs backward in time to generate samples.

The Core Idea

In the previous lesson, you saw how forward diffusion can be described as a Stochastic Differential Equation (SDE) that gradually adds noise. Now comes the magic: we can run that process backward in time to generate samples from pure noise!

The reverse-time SDE looks like this:

dx = [f(x,t) - g(t)² ∇ₓ log p_t(x)] dt + g(t) dw̄

Where:

  • f(x,t) and g(t) are from your forward SDE
  • ∇ₓ log p_t(x) is the score function (gradient of log-probability)
  • dw̄ is a backward-time Brownian motion
  • Time flows from T → 0 (backward!)

Why This Works

Think of it like filming a movie of ink dispersing in water, then playing it in reverse. The reverse process needs two things:

  1. Drift correction: The term -g(t)² ∇ₓ log p_t(x) acts like a "smart guide" that steers random noise back toward realistic data. The score function tells us which direction increases probability.

  2. Stochasticity: The g(t) dw̄ term keeps the process random, ensuring diverse samples.

The Generation Recipe

Start with pure noise x_T ~ N(0,I) at time T, then simulate the reverse SDE backward to time 0. The score function ∇ₓ log p_t(x) is what your neural network learns during training—it's the "compass" pointing toward data.

Key Takeaway: The reverse-time SDE generates samples by running diffusion backward using learned scores as a drift correction that guides noise back into realistic data.