Monte Carlo Methods and Simulation
What You'll Learn
You'll discover how the Law of Large Numbers provides the mathematical foundation for using random sampling (Monte Carlo methods) to approximate complicated expectations and integrals.
The Core Idea
Monte Carlo methods use repeated random sampling to solve problems that are difficult or impossible to tackle analytically. The Law of Large Numbers (both weak and strong forms) guarantees that as we take more and more independent samples, the sample average converges to the true expected value.
Here's the connection: Suppose you want to compute E[g(X)] for some function g and random variable X, but the integral or sum is messy. Instead of calculating it directly, you can:
- Generate many independent samples X₁, X₂, ..., Xₙ from the distribution of X
- Compute g(X₁), g(X₂), ..., g(Xₙ)
- Take their average: (1/n)∑g(Xᵢ)
By the Law of Large Numbers, this sample average converges to E[g(X)] as n → ∞.
Why This Works for Integrals
For integrals over [0,1], say ∫₀¹ h(x)dx, notice this equals E[h(U)] where U ~ Uniform(0,1). Generate uniform random samples U₁, ..., Uₙ, compute the average (1/n)∑h(Uᵢ), and you've approximated the integral! The same principle extends to higher dimensions and more complex domains using appropriate distributions.
The Analogy
Think of estimating the average height of people in a city. You can't measure everyone (the true expectation), but by randomly sampling many people and averaging their heights (Monte Carlo), you get closer and closer to the true average—that's the Law of Large Numbers at work.
Key Takeaway: The Law of Large Numbers mathematically justifies Monte Carlo simulation by guaranteeing that sample averages of independent random samples converge to the true expectation, enabling numerical approximation of otherwise intractable expectations and integrals.