Course contentsShow
← Machine Learning and Deep Learning
Lesson 1547 of 3,53834. Diffusion ModelsFree lesson

Connection to Variational Lower Bound

Understanding how the DDPM loss derives from maximizing the evidence lower bound (ELBO).

Connection to Variational Lower Bound

What You'll Learn

You'll understand how the simplified DDPM training loss you've been using actually comes from maximizing the evidence lower bound (ELBO), connecting diffusion models to the variational inference framework you learned with VAEs.

The Theoretical Foundation

Remember how VAEs maximize the ELBO to train generative models? DDPM does the same thing, but for a much longer chain of latent variables (one per diffusion timestep).

The full DDPM objective is derived by writing out the ELBO for the entire forward and reverse Markov chain. When you expand this variational bound, you get a sum of terms—one for each timestep—that measure how well your reverse process matches the true posterior distribution.

From ELBO to Simplified Loss

The mathematical derivation shows that maximizing the ELBO is equivalent to minimizing a weighted sum of denoising errors across all timesteps. Each term compares your predicted noise to the actual noise added at that step.

The "simplified loss" you learned earlier—predicting the noise ε at each timestep—is actually a reweighted version of this ELBO. The original derivation includes coefficients that depend on the variance schedule, but empirically, dropping these weights (treating all timesteps equally) works better in practice.

Think of it this way: Just as VAEs use ELBO to bridge exact likelihood (intractable) with a trainable bound, DDPMs use ELBO to connect the ideal reverse diffusion process (unknown) with your learned denoising network. The simplified loss is just the most practical form of this bound for training.

Key Takeaway: The DDPM training objective is theoretically grounded in maximizing the variational lower bound across the entire diffusion chain, with the simplified noise prediction loss being an empirically effective reweighting of this ELBO.