Proof Outline Using Chebyshev's Inequality
You'll learn how Chebyshev's inequality rigorously proves the Weak Law of Large Numbers for i.i.d. random variables with finite variance.
The Core Idea
The Weak Law of Large Numbers says that the sample mean converges in probability to the expected value. But how do we prove this? Chebyshev's inequality gives us the perfect tool.
Recall that Chebyshev's inequality tells us:
P(|X - μ| ≥ ε) ≤ Var(X) / ε²
Now suppose we have i.i.d. random variables X₁, X₂, ..., Xₙ, each with mean μ and variance σ². Let X̄ₙ = (X₁ + ... + Xₙ)/n be their sample mean.
The Proof Outline
Step 1: Find E[X̄ₙ] and Var(X̄ₙ)
- E[X̄ₙ] = E[(X₁ + ... + Xₙ)/n] = nμ/n = μ
- Var(X̄ₙ) = Var((X₁ + ... + Xₙ)/n) = (1/n²) · nσ² = σ²/n
(The variance calculation uses independence to sum variances.)
Step 2: Apply Chebyshev's inequality to X̄ₙ
For any ε > 0:
P(|X̄ₙ - μ| ≥ ε) ≤ Var(X̄ₙ) / ε² = σ² / (nε²)
Step 3: Take the limit as n → ∞
As n grows, σ²/(nε²) → 0. Therefore:
lim (n→∞) P(|X̄ₙ - μ| ≥ ε) = 0
This is exactly the definition of convergence in probability: X̄ₙ →ᵖ μ.
Why This Works
The variance of the sample mean shrinks as we average more variables. Chebyshev bounds the probability of deviations by this shrinking variance, forcing that probability toward zero.
Key Takeaway: Chebyshev's inequality proves the Weak Law by showing that P(|X̄ₙ - μ| ≥ ε) ≤ σ²/(nε²) → 0, establishing convergence in probability.