Population vs Sample
What you'll learn: You'll understand the crucial difference between a population (the entire group) and a sample (a smaller piece of it), and why this matters for drawing conclusions in machine learning.
The Core Idea
Imagine you want to know the average height of all adults in your country. Measuring everyone would be impossible—that's your population: the complete set of all individuals or observations you're interested in studying.
Instead, you measure a manageable group—say, 1,000 people. This smaller group is your sample: a subset drawn from the population. From this sample, you calculate statistics (like average height) and use them to estimate what's true for the entire population.
Why This Distinction Matters
- Population parameters are the true values (mean, variance, etc.) that describe the entire population. We rarely know these—they're what we're trying to discover.
- Sample statistics are the values we calculate from our sample data. These are our estimates of the population parameters.
The Law of Large Numbers and Central Limit Theorem (which you just learned!) explain why sample statistics become reliable estimates as sample size grows.
In Machine Learning Context
When you train a model, your training data is a sample of all possible data points in the real world (the population). You compute statistics from this sample (mean pixel values, feature correlations) and hope they represent the broader population your model will encounter in production.
Key Takeaway: A population is the complete set you want to understand; a sample is the smaller subset you actually observe. Sample statistics estimate population parameters, and larger samples generally give better estimates.