Course contentsShow
← Machine Learning and Deep Learning
Lesson 1563 of 3,53834. Diffusion ModelsFree lesson

Numerical Solvers for Sampling

Explore numerical methods for solving reverse SDEs and ODEs, including Euler-Maruyama and higher-order solvers.

Numerical Solvers for Sampling

What you'll learn: How different numerical methods trade off speed and accuracy when simulating the reverse diffusion process for sample generation.

Why Numerical Solvers Matter

In diffusion models, sampling requires solving either a reverse-time stochastic differential equation (SDE) or a probability flow ordinary differential equation (ODE) backward through time. These equations don't have closed-form solutions, so we need numerical methods to approximate them step-by-step.

Think of it like navigating from a noisy image back to a clear one: you can take many tiny, careful steps (slow but accurate) or fewer, larger leaps (fast but less precise).

The Euler-Maruyama Method

The Euler-Maruyama solver is the simplest approach for SDEs. It's essentially the stochastic version of Euler's method:

x_{t-Δt} = x_t + drift(x_t, t) · Δt + diffusion(x_t, t) · √Δt · z

where z is random noise. This is what ancestral sampling in DDPM uses implicitly. It's straightforward but requires many small timesteps for accuracy.

Higher-Order Solvers

Higher-order methods like Heun's method, Runge-Kutta solvers, or the DPM-Solver evaluate the model multiple times per step to estimate trajectories more accurately. They can:

  • Take larger timesteps while maintaining quality
  • Generate samples in 10-50 steps instead of 1000
  • Trade computation per step for fewer total steps

For ODEs (probability flow), deterministic solvers like RK45 are often used since there's no stochastic component.

The Speed-Quality Trade-off

Euler-Maruyama is simple and stable but slow. Advanced solvers generate comparable quality in 10× fewer steps, dramatically speeding up sampling—critical for practical applications.

Key Takeaway: Numerical solvers determine how efficiently you can sample from diffusion models by approximating the reverse SDE/ODE; higher-order methods enable faster generation with fewer evaluation steps.