Course contentsShow
← Machine Learning and Deep Learning
Lesson 1443 of 3,53832. Generative Models: Autoencoders and VAEsFree lesson

The Reparameterization Trick

Master the trick that enables backpropagation through random sampling: z = μ + σ ⊙ ε where ε ~ N(0,I).

The Reparameterization Trick

What you'll learn: How to make random sampling differentiable so we can train VAEs with backpropagation.

The Problem

In the previous lesson, you learned that VAE encoders output probability distributions (mean μ and standard deviation σ) instead of fixed vectors. To get a latent vector z, we need to sample from that distribution. But here's the catch: sampling is a random operation, and you can't backpropagate through randomness. Gradients need a deterministic path!

The Trick

The reparameterization trick cleverly separates the randomness from the learnable parameters:

z = μ + σ ⊙ ε    where ε ~ N(0, I)

Instead of sampling directly from N(μ, σ), we:

  1. Sample ε (epsilon) from a standard normal distribution N(0,1) — this is pure noise, independent of our network
  2. Scale it by σ and shift by μ — these ARE learnable parameters from our encoder

Why It Works

Think of it like adjusting a radio dial. The static (ε) is random and unchangeable, but the tuning knobs (μ and σ) control what signal you get. Gradients can flow through μ and σ because they're just regular operations (addition and multiplication), while the randomness is isolated in ε.

Now backpropagation works! The gradient bypasses the random sampling and flows directly through μ and σ, allowing your encoder to learn the right distribution parameters.

Key Takeaway: The reparameterization trick makes sampling differentiable by moving randomness outside the computational graph: sample standard noise ε, then deterministically transform it with learnable μ and σ.