Unifying Score-Based and DDPM Perspectives
What You'll Learn
You'll discover that DDPM and score-based diffusion models are actually the same fundamental framework, just expressed in different mathematical languages.
The Core Connection
Think of DDPM and score-based models as two people describing the same journey—one uses discrete step-by-step directions, while the other uses a continuous GPS route.
DDPM (Denoising Diffusion Probabilistic Models) operates in discrete time steps. At each timestep t, you add a fixed amount of noise, then learn to reverse that specific step. The model learns p(x_{t-1} | x_t).
Score-based models work with continuous time. They model the entire diffusion as a smooth stochastic differential equation (SDE), learning the score function ∇log p_t(x) at every continuous time point.
The Unification
Here's the key insight: the noise prediction network in DDPM is directly related to the score function in score-based models. Specifically:
score ≈ -noise_prediction / sqrt(noise_variance)
When you train a DDPM to predict noise ε, you're implicitly learning the score function. Both approaches:
- Add Gaussian noise progressively (forward process)
- Learn to reverse that process (backward/reverse process)
- Use neural networks conditioned on time
- Sample by iteratively removing noise
The difference is merely discretization: DDPM uses 1000 steps (discrete), while score-based SDEs use infinitely many steps (continuous limit). The probability flow ODE you learned bridges these perspectives—it's the deterministic continuous path through the same space DDPM traverses stochastically.
Key Takeaway: DDPM and score-based models are equivalent frameworks; DDPM is the discrete-time version, and score-based SDEs are the continuous-time limit, both learning to reverse a noising process through related network objectives.