Course contentsShow
← Data Science
Lesson 1589 of 2,14538. Bayesian Data AnalysisFree lesson

Markov Chains: The Foundation of MCMC

Learn what makes a chain Markov, stationary distributions, and why they converge to target posteriors.

Markov Chains: The Foundation of MCMC

What you'll learn: You'll understand what makes a chain "Markov," how it reaches a stationary distribution, and why this mathematical property lets us sample from complex posterior distributions.

What Is a Markov Chain?

A Markov chain is a sequence of states where the next state depends only on the current state, not on how you got there. Think of it like exploring a city where your next street choice depends only on your current intersection, not the full path you walked to arrive.

Mathematically, this "memoryless" property means: P(next state | current state, all past states) = P(next state | current state).

The Stationary Distribution

If you run a Markov chain long enough, it often "forgets" where it started and settles into a predictable pattern called the stationary distribution. This is a probability distribution that doesn't change even as the chain continues moving.

Analogy: Imagine shuffling cards by repeatedly swapping random pairs. After enough swaps, the deck reaches a "thoroughly shuffled" state—the stationary distribution—regardless of the starting order.

Why This Matters for MCMC

In Bayesian analysis, when conjugate priors aren't available, we can't compute the posterior distribution directly. Instead, we design a Markov chain whose stationary distribution is the posterior we want.

The chain explores the parameter space, spending more time in high-probability regions. After a "burn-in" period (discarding early samples before convergence), the remaining samples approximate draws from the posterior distribution.

Key conditions for convergence:

  • Irreducibility: The chain can eventually reach any state from any other state
  • Aperiodicity: The chain doesn't get stuck in cycles
  • Ergodicity: Long-run averages converge to expectations under the stationary distribution

Key Takeaway: A Markov chain's memoryless property and convergence to a stationary distribution let us generate samples from complex posterior distributions that we can't compute directly—this is the mathematical foundation of MCMC methods.