β-VAE for Disentanglement
What you'll learn: How tuning a single hyperparameter (β) in VAEs can encourage the model to learn independent, interpretable factors in its latent space.
The Core Idea
In a standard VAE, the loss balances reconstruction quality against KL divergence regularization with equal weight. β-VAE modifies this by multiplying the KL term by a factor β > 1:
Loss = Reconstruction Loss + β × KL Divergence
When β is larger than 1, you're telling the model: "Care more about keeping the latent variables close to independent Gaussians, even if it costs you a bit in reconstruction quality."
Why This Encourages Disentanglement
Think of the latent space like a filing cabinet. In a regular VAE, files might be messily organized—information about color, shape, and position all mixed together in overlapping drawers.
By increasing β, you force the model to use its limited "information budget" more carefully. The stronger KL penalty pushes each latent dimension to be independent and selective—one dimension might capture only rotation, another only color, another only size. This separation of factors is called disentanglement.
The Trade-off
Higher β values (like β = 4 or β = 10) create cleaner, more interpretable latent spaces where you can smoothly vary individual attributes. However, reconstruction quality typically decreases because the model has less freedom to encode complex correlations.
Key Takeaway: β-VAE weights the KL divergence term with β > 1 to pressure the model into learning disentangled latent representations where each dimension captures an independent factor of variation, at the cost of some reconstruction fidelity.