Central Limit Theorem
What You'll Learn
You'll discover why averaging multiple samples produces a predictable pattern, even when the original data looks nothing like a bell curve.
The Core Idea
The Central Limit Theorem (CLT) is one of probability's most powerful results: when you take many random samples from any distribution and calculate their means, those sample means will form a normal (Gaussian) distribution—regardless of the original distribution's shape.
This seems almost magical. Imagine rolling a die thousands of times. Each roll follows a uniform distribution (every number 1-6 equally likely—definitely not bell-shaped). But if you repeatedly take groups of, say, 30 rolls and calculate each group's average, those averages will cluster around 3.5 in a beautiful normal curve.
Why It Matters
The CLT explains why normal distributions appear everywhere in nature and machine learning. When many independent factors combine (like measurement errors, biological variations, or user behaviors), their combined effect tends toward normality.
For practical work:
- Sample size matters: Larger samples (typically n ≥ 30) produce better normal approximations
- The original distribution becomes irrelevant: Exponential, uniform, bimodal—doesn't matter
- Predictable spread: The standard deviation of sample means equals the population standard deviation divided by √n
An Analogy
Think of tossing coins. One coin flip is purely binary (heads or tails). But average 100 flips together, and you'll consistently get around 50% heads—with results forming a predictable bell curve across many trials.
Key Takeaway: The Central Limit Theorem guarantees that sample means follow a normal distribution as sample size increases, making statistical inference possible even when the underlying population distribution is unknown or non-normal.