GP Prior: Sampling Functions
You'll learn how Gaussian Processes define a distribution over entire functions before seeing any data, and how to visualize what these functions look like.
What Is a GP Prior?
In Gaussian Process regression, we don't just model parameters—we model entire functions. Before observing any training data, a GP defines a prior belief about what functions are plausible. This prior is controlled entirely by your choice of kernel function (which you learned about in previous lessons).
Think of it like this: imagine you have an infinite bag of possible functions. The GP prior tells you which functions are more likely to be drawn from that bag. Functions that are smooth and continuous (if using an RBF kernel) are more probable than wildly oscillating ones.
Visualizing the Prior
When you sample from a GP prior, you're literally drawing random functions from this distribution. Each sample is a complete function from input to output. If you sample multiple times, you get different functions, but they all share characteristics defined by your kernel—like smoothness, periodicity, or roughness.
For example, with an RBF kernel with specific length-scale and variance parameters, you might sample five functions. They'll all be smooth curves, but each will wiggle differently. None have seen data yet—they're pure "guesses" about what the underlying function might look like.
Why This Matters
Understanding the prior helps you choose appropriate kernels and hyperparameters. If your sampled functions look nothing like what you expect your data to follow (too smooth, too wiggly, wrong periodicity), you know your prior assumptions need adjustment.
Key Takeaway:
The GP prior is a distribution over functions defined by your kernel choice; sampling from it visualizes your model's beliefs before seeing any data, helping you validate modeling assumptions.