Course contentsShow
← Machine Learning and Deep Learning
Lesson 580 of 3,53814. Probabilistic and Bayesian Machine LearningFree lesson

Conjugate Priors and Analytical Posteriors

Understand conjugate prior families that yield closed-form posterior distributions, enabling exact Bayesian inference.

Conjugate Priors and Analytical Posteriors

What you'll learn: How choosing the right prior family lets you compute exact posterior distributions without numerical approximation.

The Core Idea

In Bayesian inference, you start with a prior distribution over parameters, observe data (the likelihood), and use Bayes' rule to compute the posterior distribution. Normally, this involves tricky integrals that require approximation methods. But for certain special prior-likelihood pairs, the posterior has the same distributional form as the prior—these are called conjugate priors.

Think of it like LEGO blocks that snap together perfectly: if your prior and likelihood are "compatible shapes," the resulting posterior is also the same shape, just with updated parameters.

Why This Matters

When a prior is conjugate to the likelihood:

  • The posterior can be computed in closed form (exact formula, no sampling needed)
  • You get analytical expressions for posterior parameters
  • Updates are fast and exact—perfect for online learning or real-time inference

Classic Examples

  • Beta-Binomial conjugacy: If you have a Beta prior on probability and observe Binomial data (coin flips), the posterior is also Beta
  • Gaussian-Gaussian conjugacy: With a Gaussian prior on the mean and Gaussian likelihood, the posterior mean is also Gaussian
  • Gamma-Poisson conjugacy: Gamma prior + Poisson likelihood → Gamma posterior

The Trade-off

Conjugate priors are mathematically convenient but might not perfectly capture your true prior beliefs. You're choosing analytical tractability over complete modeling flexibility.

Key Takeaway: Conjugate priors let you perform exact Bayesian inference with closed-form posteriors, avoiding expensive numerical methods—but only for specific prior-likelihood combinations that "close under" Bayes' rule.