Posterior Mean and Mode
You'll learn how to extract point estimates from a posterior distribution and see how Bayesian inference naturally "shrinks" estimates toward your prior beliefs.
What Are Point Estimates from the Posterior?
After computing a posterior distribution using Bayes' theorem, you often need a single "best guess" value rather than a full distribution. The two most common choices are:
- Posterior Mean: The expected value of the posterior distribution — think of it as the "center of mass" of your updated beliefs.
- Posterior Mode: The peak of the posterior distribution, also called the Maximum A Posteriori (MAP) estimate — the single most probable value.
Calculating These Estimates
For a Beta-Binomial model (proportion problems), if your posterior is Beta(α, β):
- Posterior Mean =
α / (α + β) - Posterior Mode =
(α - 1) / (α + β - 2)(when α, β > 1)
These are simple arithmetic, but they carry deep meaning.
The Shrinkage Effect
Here's the magic: Bayesian point estimates shrink toward the prior. If you start with a prior Beta(a, b) and observe s successes in n trials, your posterior mean becomes (a + s) / (a + b + n).
With little data, the prior dominates (strong shrinkage). With lots of data, the observed proportion dominates (weak shrinkage). This automatic regularization prevents overfitting to small samples — unlike a purely frequentist maximum likelihood estimate that ignores prior knowledge entirely.
Key Takeaway: The posterior mean and mode give you single-number summaries of your updated beliefs, with the mean naturally balancing prior knowledge and observed data through shrinkage toward the prior.