Unknown Variance: Normal-Inverse-Gamma Model
You'll learn how to perform Bayesian inference on normally distributed data when both the mean and variance are unknown by using a conjugate prior pair.
The Problem with Unknown Variance
So far, when working with normal data, you've assumed the variance was known and only estimated the mean. But in real-world scenarios, you rarely know the population variance beforehand—you must estimate both parameters simultaneously from your data.
The Normal-Inverse-Gamma Solution
The Normal-Inverse-Gamma (NIG) distribution is a conjugate prior for the normal likelihood when both mean (μ) and variance (σ²) are unknown. It's actually a joint prior distribution over both parameters:
- The Inverse-Gamma part models uncertainty about σ²
- The Normal part models uncertainty about μ (conditional on σ²)
Think of it like this: you're uncertain about how spread out your data is (variance), and that uncertainty about spread affects your uncertainty about where the center is (mean). The NIG prior captures this interdependence elegantly.
How It Works
When you observe normal data, the NIG prior updates to an NIG posterior through closed-form equations. The posterior parameters combine:
- Your prior beliefs about μ and σ²
- Information from the observed data (sample mean and variance)
- The sample size (which determines how much weight the data gets)
The result? Updated distributions for both mean and variance that properly account for uncertainty in each.
Why This Matters
Many real scenarios require estimating both parameters: measuring customer lifetime value, quality control processes, or response times. The NIG model lets you do full Bayesian inference without computationally expensive sampling methods.
Key Takeaway: When both mean and variance are unknown for normal data, the Normal-Inverse-Gamma conjugate prior enables tractable Bayesian inference by jointly modeling uncertainty in both parameters and updating them together as data arrives.