Course contentsShow
← Machine Learning and Deep Learning
Lesson 572 of 3,53814. Probabilistic and Bayesian Machine LearningFree lesson

GP Posterior: Conditioning on Data

Deriving the posterior distribution over functions given observed data points using Gaussian conditioning.

GP Posterior: Conditioning on Data

What you'll learn: How observed data points transform a Gaussian Process prior into a refined posterior distribution that makes predictions.

The Core Idea

Remember that a GP prior represents all possible functions we might believe in before seeing any data. Once we observe actual data points, we need to update our beliefs. This is called conditioning — we're restricting our infinite family of functions to only those that pass through (or near) our observed points.

Think of it like this: before data, you're imagining all possible paths through a forest. After seeing footprints at specific locations, you narrow down to only paths that pass through those exact spots.

Mathematical Conditioning

When you condition a Gaussian Process on observed data (X_train, y_train), you use Gaussian conditioning rules to derive the posterior:

  1. Joint distribution: Your GP prior defines a joint distribution over training outputs y_train and test outputs y_test
  2. Conditioning formula: Given observations, the posterior mean becomes a weighted combination of your prior mean and the data, smoothed by the kernel
  3. Updated uncertainty: The posterior covariance shrinks near observed points — you're more confident where you have data

The posterior at new test point x* has:

  • Mean: Prediction incorporating observed values via kernel similarities
  • Variance: Reduced uncertainty, smallest near training points

Key Takeaway

GP posterior conditioning uses Gaussian properties to update the prior distribution, producing a refined function distribution that respects observed data while maintaining smooth predictions and quantified uncertainty everywhere else.