Maximum A Posteriori Estimation
You'll learn how to find the single "best" weight estimate from a Bayesian posterior, and discover its surprising connection to regularization.
What Is MAP Estimation?
In Bayesian inference, you've learned that the posterior distribution P(w|D) captures all our updated beliefs about weights w given data D. But sometimes you need just one number—a point estimate—instead of an entire distribution.
Maximum A Posteriori (MAP) estimation finds the weight values that have the highest probability under the posterior. Mathematically, you're looking for:
w_MAP = argmax_w P(w|D)
This is the mode of the posterior distribution—the peak of the curve.
The Connection to Regularization
Here's where it gets interesting. Using Bayes' rule:
P(w|D) ∝ P(D|w) × P(w)
Taking the logarithm (which preserves the maximum):
log P(w|D) = log P(D|w) + log P(w) + constant
log P(D|w)is your likelihood (how well weights fit the data)log P(w)is your prior (your regularization preference!)
If you use a Gaussian prior on weights (common choice), log P(w) becomes proportional to -λ||w||². Maximizing MAP becomes:
maximize: log P(D|w) - λ||w||²
This is exactly Ridge regression (L2-regularized regression)! The regularization penalty is the prior distribution in disguise.
Analogy
Think of MAP as finding the highest point on a hill (posterior), where the hill's shape combines both terrain quality (likelihood) and your preferences about where peaks should be (prior).
Key Takeaway: MAP estimation finds the posterior's mode as a single best guess; with a Gaussian prior, MAP is equivalent to L2-regularized maximum likelihood, revealing that regularization is Bayesian reasoning.