Course contentsShow
← Mathematics
Lesson 2976 of 3,12158. Joint and Limit BehaviorFree lesson

Monte Carlo Methods and Simulation

Use the Law of Large Numbers to justify Monte Carlo simulation for approximating expectations and integrals.

Monte Carlo Methods and Simulation

What You'll Learn

You'll discover how the Law of Large Numbers provides the mathematical foundation for using random sampling (Monte Carlo methods) to approximate complicated expectations and integrals.

The Core Idea

Monte Carlo methods use repeated random sampling to solve problems that are difficult or impossible to tackle analytically. The Law of Large Numbers (both weak and strong forms) guarantees that as we take more and more independent samples, the sample average converges to the true expected value.

Here's the connection: Suppose you want to compute E[g(X)] for some function g and random variable X, but the integral or sum is messy. Instead of calculating it directly, you can:

  1. Generate many independent samples X₁, X₂, ..., Xₙ from the distribution of X
  2. Compute g(X₁), g(X₂), ..., g(Xₙ)
  3. Take their average: (1/n)∑g(Xᵢ)

By the Law of Large Numbers, this sample average converges to E[g(X)] as n → ∞.

Why This Works for Integrals

For integrals over [0,1], say ∫₀¹ h(x)dx, notice this equals E[h(U)] where U ~ Uniform(0,1). Generate uniform random samples U₁, ..., Uₙ, compute the average (1/n)∑h(Uᵢ), and you've approximated the integral! The same principle extends to higher dimensions and more complex domains using appropriate distributions.

The Analogy

Think of estimating the average height of people in a city. You can't measure everyone (the true expectation), but by randomly sampling many people and averaging their heights (Monte Carlo), you get closer and closer to the true average—that's the Law of Large Numbers at work.

Key Takeaway: The Law of Large Numbers mathematically justifies Monte Carlo simulation by guaranteeing that sample averages of independent random samples converge to the true expectation, enabling numerical approximation of otherwise intractable expectations and integrals.