Course contentsShow
← Machine Learning and Deep Learning
Lesson 1544 of 3,53834. Diffusion ModelsFree lesson

The Denoising Network Architecture

U-Net with time embeddings as the standard architecture for predicting noise at each timestep.

The Denoising Network Architecture

What you'll learn: How U-Net with time embeddings became the standard architecture for predicting noise in diffusion models.

The Core Challenge

In diffusion models, the reverse process needs a neural network that can look at a noisy image and predict what noise was added to it. But here's the twist: the network must do this for images at different noise levels. An image that's 10% noisy requires different denoising than one that's 90% corrupted.

Why U-Net?

The U-Net architecture emerged as the go-to choice for this task. Originally designed for medical image segmentation, U-Net has a distinctive structure:

  • Encoder path: Gradually downsamples the input, extracting hierarchical features
  • Decoder path: Upsamples back to original resolution
  • Skip connections: Copy features from encoder to decoder at matching resolutions

Think of it like restoring an old photograph: you need both the big picture (what object this is) and fine details (texture, edges). Skip connections let the network preserve spatial details while understanding global context.

Time Embeddings: The Secret Sauce

The crucial innovation for diffusion models: time embeddings. The network receives:

  1. The noisy image
  2. A timestep value (e.g., t=500 out of 1000 steps)

The timestep gets embedded (similar to positional encodings in Transformers) and injected into the U-Net at multiple layers. This tells the network "how noisy" the input is, so it can adapt its denoising strategy accordingly.

Analogy: Like a photo editor where you tell the software not just what to fix, but also how damaged the photo is.

Key Takeaway: U-Net with time embeddings provides the ideal architecture for diffusion models, combining spatial understanding through skip connections with timestep-aware denoising through injected time embeddings.