Predictive Distributions: Prior and Posterior Predictive
You'll learn how to make predictions about future observations by accounting for uncertainty in the parameters themselves.
The Core Idea
When making predictions in a Bayesian framework, you don't just plug in a single "best guess" parameter value. Instead, you integrate over all possible parameter values, weighted by how plausible each value is. This gives you a distribution for future data, not just a point estimate.
There are two flavors:
Prior Predictive Distribution
Before seeing any data, you use your prior distribution to predict what future observations might look like. You're essentially asking: "What data would I expect if I averaged over all my prior beliefs about the parameters?"
Formula: If θ is your parameter and y is a future observation,
p(y) = ∫ p(y | θ) p(θ) dθ
This marginalizes out θ by weighting the likelihood with your prior.
Posterior Predictive Distribution
After observing data D, you update to the posterior distribution for θ, then use that to predict new observations y*:
p(y* | D) = ∫ p(y* | θ) p(θ | D) dθ
Now your predictions incorporate what you've learned from the data.
Why It Matters
This is how Bayesian models make honest predictions: they acknowledge parameter uncertainty rather than pretending you know θ exactly. It's like asking a panel of experts (each with a different parameter value) to vote on what happens next, weighted by their credibility.
Key Takeaway: Predictive distributions integrate the likelihood over prior (before data) or posterior (after data) uncertainty, producing realistic forecasts that reflect both data variability and parameter uncertainty.