Course contentsShow
← Machine Learning and Deep Learning
Lesson 579 of 3,53814. Probabilistic and Bayesian Machine LearningFree lesson

Exact Inference: Marginalization and Conditioning

Learn how to compute exact posterior distributions through marginalization and conditioning in simple Bayesian models.

Exact Inference: Marginalization and Conditioning

What You'll Learn

You'll learn how to compute exact posterior distributions in simple Bayesian models by applying the fundamental operations of marginalization and conditioning.

The Core Idea

In Bayesian machine learning, we often have joint probability distributions over multiple variables. Exact inference means computing probabilities of interest without approximation, using two key operations:

Marginalization is like "summing out" or "integrating out" variables you don't care about. Think of it as collapsing a multi-dimensional probability table down to fewer dimensions by adding up all the possibilities for unwanted variables.

Conditioning means updating your beliefs given observed evidence. When you condition on data, you're restricting your probability distribution to only the scenarios consistent with what you've observed, then renormalizing.

How They Work Together

In Bayesian models like those you've seen with Gaussian Processes or Bayesian Linear Regression, you typically:

  1. Start with a joint distribution P(parameters, data)
  2. Condition on observed data to get P(parameters | data) — this is your posterior
  3. Marginalize over parameters to make predictions: P(new_data | observed_data)

For example, in a GP posterior (which you learned in lesson 572), conditioning on training data gives you the posterior distribution over functions. Then marginalizing over all possible functions gives you the predictive distribution at test points.

These operations are "exact" when you can compute them analytically (like with conjugate priors) or with direct numerical integration in low dimensions. They become intractable for complex models, which is why approximate inference methods exist.

Key Takeaway: Exact inference uses marginalization to eliminate nuisance variables and conditioning to incorporate observed data, allowing you to compute exact posterior and predictive distributions in simple Bayesian models where analytical or direct numerical solutions are possible.