Mean-Field Variational Inference
What you'll learn: How to approximate complex posterior distributions by assuming independence between variables and deriving efficient update rules.
The Core Idea
In Bayesian inference, we want the posterior distribution p(θ|data), but it's often intractable to compute exactly. Mean-field variational inference simplifies this by assuming the posterior can be factorized into independent components:
q(θ) = q₁(θ₁) × q₂(θ₂) × ... × qₙ(θₙ)
This is like assuming each parameter lives in its own world, unaware of correlations with others. While this seems overly simple, it makes inference computationally feasible.
The Factorization Assumption
Instead of modeling the full joint posterior (which might have complex dependencies), mean-field VI assumes each variable θⱼ has its own independent distribution. Think of it like weather forecasting: rather than modeling every interaction between temperature, pressure, and humidity simultaneously, you model each factor separately and combine predictions.
Coordinate Ascent Updates via ELBO
Remember the ELBO (Evidence Lower Bound) from variational inference? It measures how well q(θ) approximates the true posterior. Mean-field VI optimizes each factor qⱼ(θⱼ) one at a time while holding others fixed—this is called coordinate ascent.
For each factor, the optimal update is:
log qⱼ*(θⱼ) = 𝔼_{q₋ⱼ}[log p(θ, data)] + constant
Where 𝔼_{q₋ⱼ} means "expectation over all other factors." You cycle through all variables, updating each based on the current values of others, iteratively climbing toward a better ELBO.
The Trade-off
Mean-field approximations are fast and scalable but sacrifice accuracy by ignoring correlations. They work best when variables are genuinely weakly correlated.
Key Takeaway: Mean-field variational inference assumes a factorized posterior for tractability, optimizing each factor independently via coordinate ascent on the ELBO—trading correlation modeling for computational efficiency.