Course contentsShow
← Machine Learning and Deep Learning
Lesson 88 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Bootstrap Resampling

Estimating sampling distributions and confidence intervals through resampling with replacement.

Bootstrap Resampling

What you'll learn: How to estimate the uncertainty of a statistic by repeatedly resampling your data with replacement.

The Core Idea

Imagine you have a small dataset and you want to know how confident you should be in your sample mean or any other statistic. Ideally, you'd collect hundreds of new samples from the population, but that's often impossible or expensive. Bootstrap resampling offers a clever alternative: treat your sample as a mini-population and draw new samples from it—with replacement.

"With replacement" means after you pick a data point, you put it back before picking again. This allows the same observation to appear multiple times in a resample (or not at all), creating realistic variability.

How It Works

  1. Start with your original sample of size n
  2. Draw a new sample of size n by randomly selecting observations with replacement
  3. Calculate your statistic (mean, median, etc.) on this resample
  4. Repeat steps 2-3 many times (often 1,000 or 10,000 times)
  5. Examine the distribution of your statistic across all resamples

This creates a sampling distribution that estimates the variability of your statistic. You can then construct confidence intervals by finding percentiles (e.g., the 2.5th and 97.5th percentiles for a 95% confidence interval).

Practical Analogy

Think of your sample as a bag of marbles. Instead of finding more marbles in the world, you draw marbles from your bag, note the color, put it back, shake, and repeat. After many draws, you understand the variety in your bag—and by extension, the population it represents.

Key Takeaway: Bootstrap resampling estimates uncertainty and confidence intervals by repeatedly sampling with replacement from your data, creating an empirical sampling distribution without needing the true population.