Burn-in, Thinning, and Convergence Diagnostics
What you'll learn: How to ensure your MCMC samples accurately represent the posterior distribution by discarding warm-up samples, reducing autocorrelation, and checking chain convergence.
The Problem with Raw MCMC Samples
When you run MCMC algorithms like Metropolis-Hastings or Gibbs sampling, the chain doesn't immediately explore the posterior distribution correctly. It starts from an arbitrary initial value and needs time to "find" the high-probability regions. Additionally, consecutive samples are often highly correlated, which can mislead your inference.
Burn-in: Discarding the Warm-Up Period
Burn-in refers to discarding the first portion of your MCMC samples—typically the first 10-50% of iterations. Think of it like letting your car engine warm up before driving: those initial samples are still "searching" for the right distribution and shouldn't be trusted for inference.
Thinning: Reducing Autocorrelation
Thinning means keeping only every kth sample (e.g., every 10th) to reduce correlation between consecutive draws. If your samples are highly autocorrelated, they contain redundant information. Thinning creates a more independent set of samples, though it reduces your effective sample size.
Convergence Diagnostics: Did It Work?
You must verify that your chain has actually converged to the posterior. Common diagnostics include:
- Trace plots: Visualize the chain over iterations—it should look like random noise around a stable mean, not trending or stuck
- R-hat statistic: Compares variance within and between multiple chains; values near 1.0 indicate convergence
- Effective sample size (ESS): Estimates how many independent samples you truly have after accounting for autocorrelation
Key Takeaway: MCMC samples require post-processing—discard burn-in samples to eliminate initialization bias, consider thinning for autocorrelation, and always check convergence diagnostics before trusting your posterior inference.