GP Posterior: Conditioning on Data
What you'll learn: How observed data points transform a Gaussian Process prior into a refined posterior distribution that makes predictions.
The Core Idea
Remember that a GP prior represents all possible functions we might believe in before seeing any data. Once we observe actual data points, we need to update our beliefs. This is called conditioning — we're restricting our infinite family of functions to only those that pass through (or near) our observed points.
Think of it like this: before data, you're imagining all possible paths through a forest. After seeing footprints at specific locations, you narrow down to only paths that pass through those exact spots.
Mathematical Conditioning
When you condition a Gaussian Process on observed data (X_train, y_train), you use Gaussian conditioning rules to derive the posterior:
- Joint distribution: Your GP prior defines a joint distribution over training outputs
y_trainand test outputsy_test - Conditioning formula: Given observations, the posterior mean becomes a weighted combination of your prior mean and the data, smoothed by the kernel
- Updated uncertainty: The posterior covariance shrinks near observed points — you're more confident where you have data
The posterior at new test point x* has:
- Mean: Prediction incorporating observed values via kernel similarities
- Variance: Reduced uncertainty, smallest near training points
Key Takeaway
GP posterior conditioning uses Gaussian properties to update the prior distribution, producing a refined function distribution that respects observed data while maintaining smooth predictions and quantified uncertainty everywhere else.