How Bootstrap Resampling Works
You'll learn the step-by-step mechanics of creating bootstrap samples and why sampling with replacement produces valid statistical estimates.
The Core Idea
Bootstrap resampling is beautifully simple: you repeatedly draw samples from your original sample, each time selecting observations with replacement. This means after you pick an observation, you put it back in the pool, so it might get selected again (or not at all) in the same resample.
Here's the process:
- Start with your original sample of size n
- Randomly select one observation from it
- Record that observation in your bootstrap sample
- Put it back (this is the "with replacement" part)
- Repeat steps 2-4 until you have n observations in your bootstrap sample
- Calculate the statistic you care about (mean, median, etc.) for this bootstrap sample
- Repeat the entire process hundreds or thousands of times
Why Replacement Matters
Sampling with replacement creates variability that mimics the uncertainty of drawing different samples from the true population. Each bootstrap sample will have some observations repeated and others missing entirely—this randomness mirrors what would happen if you could collect new samples from the population.
Think of it like this: your original sample is now standing in as a "mini-population." By resampling from it with replacement, you're simulating what you'd see if you repeatedly sampled from the real population.
The Result
After creating many bootstrap samples (typically 1,000–10,000), you have a distribution of your statistic. This bootstrap distribution shows you the range and variability of what your estimate might be, forming the foundation for confidence intervals.
Key Takeaway: Bootstrap resampling works by repeatedly sampling n observations with replacement from your original sample, creating many pseudo-samples that capture the uncertainty inherent in your original estimate.