Posterior Mean as Weighted Average
You'll learn: How Bayesian inference elegantly balances your initial beliefs with observed data by combining them through precision-weighted averaging.
The Core Idea
When working with a conjugate Normal-Normal model (using a normal prior for a normal mean), the posterior mean isn't just "somewhere between" the prior mean and the sample mean—it's a precise weighted average of the two. The weights depend on how much information each source provides, measured by their precision (inverse variance).
The formula looks like this:
Posterior Mean = (precision_prior × prior_mean + precision_data × sample_mean)
/ (precision_prior + precision_data)
Where precision = 1/variance.
Why Precision?
Think of precision as "confidence weight." A distribution with low variance (high precision) is very confident about where the true value lies, so it gets more weight. A distribution with high variance (low precision) is uncertain, so it gets less weight.
Analogy: Imagine two friends estimating a restaurant's wait time. Friend A (been there 50 times) says "30 minutes" with high confidence. Friend B (been once) says "15 minutes" but isn't sure. You'd weight Friend A's estimate more heavily—that's exactly what Bayesian inference does mathematically.
The Balance Shifts
- Strong prior + weak data: Posterior stays close to prior mean
- Weak prior + strong data: Posterior moves toward sample mean
- Equal information: Posterior sits roughly halfway between
As you collect more data, the data precision grows (proportional to sample size), gradually overwhelming the fixed prior precision.
Key Takeaway: The posterior mean is a precision-weighted average where high-confidence information (low variance) dominates low-confidence information (high variance), creating an optimal balance between prior knowledge and observed evidence.