Course contentsShow
← Machine Learning and Deep Learning
Lesson 1449 of 3,53832. Generative Models: Autoencoders and VAEsFree lesson

VAE as Variational Inference

Understand VAEs as approximating the intractable posterior p(z|x) with a tractable q(z|x).

VAE as Variational Inference

What You'll Learn

You'll understand how VAEs use variational inference to approximate an intractable posterior distribution with a learnable, tractable one.

The Core Problem

In generative modeling, we want to know p(z|x) — the probability of latent variables z given observed data x. This tells us "what hidden representation likely generated this data?"

Unfortunately, computing p(z|x) directly requires integrating over all possible latent values:

p(z|x) = p(x|z)p(z) / p(x), where p(x) = ∫ p(x|z)p(z) dz

That integral is intractable for complex models — we can't compute it.

The Variational Solution

Variational inference solves this by approximation:

Instead of computing the true posterior p(z|x), we introduce a simpler, parametric distribution q(z|x) that we can compute and optimize. In VAEs, this is the encoder network.

Think of it like approximating a complex mountain range (true posterior) with a smooth, simple hill (q) that's close enough for practical purposes.

How VAEs Fit In

The VAE encoder outputs parameters of q(z|x) — typically mean μ and variance σ² of a Gaussian. The decoder represents p(x|z).

By minimizing the ELBO (which you've seen), we're simultaneously:

  • Making q(z|x) close to the true p(z|x) (via KL divergence)
  • Ensuring the latent codes decode well (via reconstruction loss)

This is variational inference in action: we're learning the best tractable approximation to an intractable distribution.

Key Takeaway: VAEs frame generative modeling as variational inference, replacing the intractable true posterior p(z|x) with a learnable approximate posterior q(z|x) parameterized by the encoder network.