Why MCMC? When Conjugate Priors Are Not Enough
You'll learn when Bayesian inference requires simulation methods because analytical solutions become impossible.
The Limitation of Conjugate Priors
You've learned that conjugate priors make Bayesian inference elegant—the posterior has a closed-form solution you can write down directly. But most real-world models don't have this luxury. When your likelihood function is complex, when you use non-conjugate priors for flexibility, or when you have multiple parameters with intricate dependencies, the posterior distribution becomes analytically intractable.
What Makes a Posterior Intractable?
Remember that the posterior involves dividing by the normalizing constant (the evidence). For conjugate pairs, this constant cancels out nicely. But for most models, computing this integral:
P(θ|data) = P(data|θ) × P(θ) / ∫ P(data|θ) × P(θ) dθ
...requires integrating over all possible parameter values. In high dimensions or with non-standard distributions, this integral has no closed-form solution.
Enter MCMC
Markov Chain Monte Carlo (MCMC) methods let you sample from the posterior distribution without ever computing that troublesome integral. Think of it like exploring a dark cave with a flashlight—you can't see the whole space at once, but by wandering intelligently, you can map out where the interesting features are.
Instead of a formula, you get thousands of samples that represent the posterior distribution. These samples let you compute posterior means, credible intervals, and probabilities—everything you need for inference.
Key Takeaway: When conjugate priors don't apply or models grow complex, MCMC methods become essential by approximating the posterior through intelligent sampling rather than impossible integration.