Population vs Sample Variance
What you'll learn: Why we sometimes divide by N and sometimes by N-1 when calculating variance, and what Bessel's correction means.
The Two Formulas
When you learned about variance, you saw we calculate the average of squared deviations. But there's a critical choice: what do we divide by?
- Population variance: Divide by N (total count of all values)
- Sample variance: Divide by N-1 (one less than your sample size)
Why the Difference?
Think of it this way: if you're measuring the heights of every person in your town (the entire population), you have complete information. Dividing by N gives you the true variance.
But usually you're working with a sample—a subset meant to represent the population. When you calculate the mean from your sample, that mean is slightly "closer" to your sample points than the true population mean would be. This makes your squared deviations artificially smaller, underestimating the true variance.
Bessel's Correction
Dividing by N-1 instead of N is called Bessel's correction. It compensates for that underestimation bias. By making the denominator smaller, we get a slightly larger variance estimate—one that's unbiased and more accurate for the population.
Analogy: Imagine judging a pizza's taste by trying one slice. Your judgment is based on limited info, so you need to be a bit more cautious (inflate uncertainty) than if you'd eaten the whole pizza.
When to Use Which
- Working with the entire population? Use N.
- Working with a sample to estimate population variance? Use N-1.
In practice, most real-world data science uses samples, so you'll almost always see N-1.
Key Takeaway: Use N-1 (Bessel's correction) when calculating sample variance to get an unbiased estimate of the true population variance.