Connection to Score Matching
You'll learn: How reverse diffusion models learn to denoise by estimating the gradient of the data distribution itself.
At its core, score matching is about learning the score function—the gradient of the log probability of your data distribution. Mathematically, that's ∇ₓ log p(x), which points in the direction where data becomes more likely.
Here's the beautiful insight: when you train a diffusion model to reverse the noising process, you're actually teaching it to estimate this score function at different noise levels.
Why Does This Matter?
Think of your data distribution as a landscape of hills (high probability) and valleys (low probability). The score function is like a compass that always points "uphill" toward more probable data. During reverse diffusion, at each timestep, you need to know which direction removes noise and returns toward real data. That direction? It's precisely the score!
When your denoising network predicts what noise to remove from a noisy image, it's implicitly learning: "Given this noisy version, which direction leads back to the clean data distribution?"
The Mathematical Bridge
At each diffusion timestep t, your model learns to approximate:
∇ₓₜ log p(xₜ)
This gradient tells you how to "climb" back toward probable, clean data. The training objective of diffusion models—predicting the noise that was added—is mathematically equivalent to learning this score function across all noise levels.
This connection means diffusion models don't just memorize denoising patterns; they learn the fundamental geometry of your data distribution.
Key Takeaway: Reverse diffusion training teaches your model to estimate the score function (gradient of log probability), giving it a principled way to navigate from noise back to realistic data at every step.