Central Limit Theorem
What You'll Learn
You'll discover why averaging many random samples produces a normal distribution, no matter what the original data looked like—a phenomenon that underpins countless ML methods.
The Core Idea
The Central Limit Theorem (CLT) states that when you take the sum (or average) of many independent random variables, that sum approaches a normal distribution—even if the original variables aren't normally distributed themselves.
Think of it this way: imagine rolling a single die. The outcomes (1-6) are uniformly distributed—every number is equally likely. Now roll many dice and add them up. The sum won't be uniform anymore; it clusters around the middle values, forming a bell curve.
Why This Matters
This is why the normal distribution appears everywhere in nature and machine learning:
- Sample means from any population distribution become normally distributed as sample size grows
- Noise in real-world data often results from many tiny, independent effects adding together—producing Gaussian noise
- Many statistical tests assume normality because the CLT guarantees it for large samples
Connection to Previous Concepts
Remember the Law of Large Numbers? It told us that sample means converge to the true mean. The CLT goes further: it tells us how those sample means are distributed around the true mean—they follow a normal distribution with variance that shrinks as sample size increases.
The beauty is universality: whether your original random variables follow a uniform, exponential, or binomial distribution doesn't matter. Add enough of them together, and you get a normal distribution.
Key Conditions
The CLT works when:
- Random variables are independent
- Sample size is sufficiently large (often n ≥ 30 as a rule of thumb)
- Variables have finite variance
Key Takeaway: The Central Limit Theorem explains why normal distributions are ubiquitous—sums and averages of independent random variables naturally converge to normality, regardless of their original shape.