Course contentsShow
← Machine Learning and Deep Learning
Lesson 562 of 3,53814. Probabilistic and Bayesian Machine LearningFree lesson

Posterior Predictive Distribution

Derive the distribution over predictions by marginalizing over all plausible weight values; quantify prediction uncertainty.

Posterior Predictive Distribution

What you'll learn: How to make predictions in Bayesian ML by averaging over all plausible models, capturing both data and model uncertainty.

The Core Idea

In Bayesian machine learning, we don't commit to a single "best" set of weights. Instead, after seeing data, we have a posterior distribution over weights—a probability landscape showing which weight values are plausible and how likely each is.

When making a new prediction, the posterior predictive distribution asks: "What's my prediction for new input x*, considering all plausible weight configurations?"

We answer this by marginalizing (averaging) over the posterior distribution of weights. Mathematically:

p(y* | x*, data) = ∫ p(y* | x*, w) p(w | data) dw

Where:

  • p(y* | x*, w) is the likelihood—what prediction weight configuration w makes
  • p(w | data) is our posterior belief about weights after seeing data
  • The integral averages predictions weighted by how likely each weight configuration is

Why This Matters

Unlike point estimates (single weight values), the posterior predictive distribution gives you:

  1. The expected prediction: The mean of this distribution
  2. Uncertainty quantification: The variance tells you how confident you should be

Think of it like asking five experts (each representing different plausible weight values) for their opinion, then combining their answers weighted by their credibility. If they all agree, you're confident. If they disagree wildly, your prediction has high uncertainty.

The Two Flavors of Uncertainty

  • Aleatoric uncertainty: Noise in the data itself
  • Epistemic uncertainty: Uncertainty about which model/weights are correct (captured by the posterior)

Key Takeaway: The posterior predictive distribution delivers predictions by averaging over all plausible models, naturally quantifying how uncertain your predictions are—crucial for safety-critical applications.