Course contentsShow
← Machine Learning and Deep Learning
Lesson 563 of 3,53814. Probabilistic and Bayesian Machine LearningFree lesson

Maximum A Posteriori Estimation

Find the mode of the posterior as a point estimate; relate MAP to regularized regression with L2 penalty.

Maximum A Posteriori Estimation

You'll learn how to find the single "best" weight estimate from a Bayesian posterior, and discover its surprising connection to regularization.


What Is MAP Estimation?

In Bayesian inference, you've learned that the posterior distribution P(w|D) captures all our updated beliefs about weights w given data D. But sometimes you need just one number—a point estimate—instead of an entire distribution.

Maximum A Posteriori (MAP) estimation finds the weight values that have the highest probability under the posterior. Mathematically, you're looking for:

w_MAP = argmax_w P(w|D)

This is the mode of the posterior distribution—the peak of the curve.

The Connection to Regularization

Here's where it gets interesting. Using Bayes' rule:

P(w|D) ∝ P(D|w) × P(w)

Taking the logarithm (which preserves the maximum):

log P(w|D) = log P(D|w) + log P(w) + constant
  • log P(D|w) is your likelihood (how well weights fit the data)
  • log P(w) is your prior (your regularization preference!)

If you use a Gaussian prior on weights (common choice), log P(w) becomes proportional to -λ||w||². Maximizing MAP becomes:

maximize: log P(D|w) - λ||w||²

This is exactly Ridge regression (L2-regularized regression)! The regularization penalty is the prior distribution in disguise.

Analogy

Think of MAP as finding the highest point on a hill (posterior), where the hill's shape combines both terrain quality (likelihood) and your preferences about where peaks should be (prior).

Key Takeaway: MAP estimation finds the posterior's mode as a single best guess; with a Gaussian prior, MAP is equivalent to L2-regularized maximum likelihood, revealing that regularization is Bayesian reasoning.