Stochastic Differential Equations for Diffusion
You'll learn: How to view diffusion as a continuous-time process governed by stochastic differential equations (SDEs).
What Are SDEs in Diffusion Models?
Until now, we've treated diffusion as a discrete series of steps—adding noise at timestep 1, 2, 3, etc. But what if we let the number of steps approach infinity? We get a continuous-time process described by a Stochastic Differential Equation (SDE).
An SDE models how data evolves smoothly from its original distribution into pure noise along a continuous trajectory. Instead of jumping between discrete noise levels, the data "flows" through an infinite number of infinitesimal noise additions.
The Core Idea
Think of a discrete diffusion process like climbing stairs—each step adds a fixed amount of noise. An SDE is like walking up a smooth ramp instead. At every infinitesimal moment dt, the data point moves slightly:
- A drift term pushes it in a predictable direction (shrinking toward zero)
- A diffusion term adds a tiny random perturbation (Brownian motion)
The general form looks like:
dx = f(x, t)dt + g(t)dw
Where:
xis the data at timetf(x, t)is the drift (deterministic part)g(t)controls noise intensitydwis a Wiener process (continuous random noise)
This framework unifies discrete diffusion models under one elegant mathematical theory and enables powerful reversals using probability flow ODEs.
Key Takeaway: SDEs describe diffusion as a continuous trajectory where data smoothly transitions into noise through infinitesimal random perturbations, generalizing discrete-step diffusion models into continuous time.