Conditional VAEs
What you'll learn: How to guide a VAE to generate specific types of outputs by conditioning both the encoder and decoder on class labels or other information.
The Core Idea
A standard VAE learns to encode data into a latent space and decode it back, discovering patterns on its own. But what if you want control over what gets generated? That's where Conditional VAEs (CVAEs) come in.
The key insight: feed additional information (like class labels) to both the encoder and decoder. This conditions the model on that information, allowing you to say "generate a sample that looks like class 3" or "encode this image, knowing it's a dog."
How It Works
In a regular VAE:
- Encoder:
image → latent code - Decoder:
latent code → reconstructed image
In a Conditional VAE:
- Encoder:
(image, label) → latent code - Decoder:
(latent code, label) → reconstructed image
The label (or any conditioning variable) is typically concatenated with the input image for the encoder, and with the sampled latent code for the decoder. This way, the latent space learns to represent variations within each class rather than needing to encode which class it is.
Why This Matters
Think of it like a restaurant kitchen: a regular VAE chef creates random dishes from ingredients. A conditional VAE chef takes orders — you specify "make pasta" and the latent space determines the specific sauce, presentation, and garnish, but it's guaranteed to be pasta.
At generation time, you sample from the prior and provide the desired label, enabling controlled, targeted generation.
Key Takeaway: Conditional VAEs add controllability by feeding class labels (or other conditioning information) to both encoder and decoder, enabling you to specify what kind of sample to generate while the latent space handles the variation within that category.