Course contentsShow
← Machine Learning and Deep Learning
Lesson 1555 of 3,53834. Diffusion ModelsFree lesson

Denoising Score Matching

Learn how denoising score matching trains models by predicting scores of noise-perturbed data distributions.

Denoising Score Matching

What you'll learn: How denoising score matching trains neural networks by having them predict the score (gradient of log probability) of data that's been intentionally corrupted with noise.

The Core Problem

Remember how score matching trains models to estimate the gradient of the log probability distribution? There's a computational challenge: calculating the true score matching objective requires expensive second-order derivatives. Denoising score matching offers an elegant workaround.

The Key Insight

Instead of working with clean data directly, add noise to your data first, then train your model to predict the score of this noisy data distribution. The brilliant part? This denoising objective is mathematically equivalent to the original score matching objective, but much simpler to compute!

Here's the process:

  1. Take a clean data point x
  2. Add Gaussian noise: x̃ = x + σε where ε ~ N(0, I)
  3. Train your model to predict the score of the noisy distribution: ∇log p(x̃)
  4. The loss becomes: ||s_θ(x̃) - ∇log q(x̃|x)||²

Why This Works

When you corrupt data with noise, computing the gradient of the noisy distribution becomes tractable. You're essentially training the model to "point toward" the clean data from the noisy observation—exactly what denoising requires. This score tells you which direction moves toward higher probability (cleaner data).

Connection to Diffusion Models

This is foundational for diffusion models! They apply denoising score matching at multiple noise levels, creating a hierarchy of denoisers that can reverse the diffusion process step by step.

Key Takeaway: Denoising score matching trains models to predict scores by corrupting data with noise, making the score matching objective computationally tractable while maintaining theoretical equivalence to the original formulation.