Posterior Collapse Problem
You'll learn why VAE decoders sometimes ignore the latent code entirely and how KL annealing prevents this.
What Is Posterior Collapse?
Posterior collapse happens when your VAE's decoder becomes so powerful that it learns to generate outputs without using the latent code at all. The encoder still produces latent variables, but the decoder ignores them—rendering the entire latent space useless.
This occurs because the VAE loss has two competing terms: reconstruction loss (which the decoder optimizes) and KL divergence (which regularizes the encoder). When the decoder is strong enough (say, an autoregressive model), it can achieve low reconstruction loss by ignoring the latent code entirely and just modeling the data distribution directly.
The encoder, meanwhile, finds the easiest way to minimize KL divergence: make the approximate posterior identical to the prior (usually a standard Gaussian). Result? The latent space collapses to the prior and carries no information.
How to Detect It
If your KL divergence term drops to nearly zero during training while reconstruction improves, you're likely experiencing posterior collapse. The encoder is basically saying "I'll just output the prior distribution" rather than encoding meaningful information.
Preventing It: KL Annealing
KL annealing gradually increases the weight of the KL term during training. Start with a weight near zero, allowing the decoder to learn to use the latent code before the regularization pressure kicks in:
# Simplified training pseudocode
beta = min(1.0, current_step / annealing_steps)
loss = reconstruction_loss + beta * kl_divergence
This gives the decoder time to "learn to listen" to the latent code before regularization forces the encoder toward the prior.
Key Takeaway: Posterior collapse occurs when powerful decoders ignore latent codes; KL annealing prevents this by gradually introducing regularization pressure.