Resampling and Balanced Datasets
What you'll learn: How to use resampling techniques to balance your training data and reduce bias against underrepresented groups.
The Core Problem
When your training data has unequal representation across groups (say, 90% Group A, 10% Group B), models often learn to favor the majority class. This creates unfair outcomes where minority groups receive worse predictions.
Resampling directly addresses this by modifying your dataset's composition before training.
Three Resampling Strategies
Oversampling (Minority Expansion)
Duplicate or synthesize examples from underrepresented groups until they match the majority group's size. Think of it like copying rare trading cards until your collection is balanced.
Example: If you have 1,000 majority samples and 100 minority samples, randomly duplicate minority examples until you have 1,000 of each.
Undersampling (Majority Reduction)
Remove examples from overrepresented groups to match the minority size. Like trimming your abundant flowers to match your rare ones.
Tradeoff: You lose potentially valuable training data, which may hurt overall model performance.
Synthetic Generation (SMOTE-style)
Create new artificial examples by interpolating between existing minority samples. More sophisticated than simple duplication—it generates variations rather than exact copies.
When to Use Each
- Oversampling: When you can't afford to lose majority examples (small datasets)
- Undersampling: When you have abundant data and want faster training
- Synthetic generation: When you need diversity and have sufficient computational resources
These techniques complement reweighting—resampling physically changes your dataset, while reweighting adjusts loss contributions without modifying data.
Key Takeaway: Resampling techniques (oversampling, undersampling, and synthetic generation) directly modify training data distribution to achieve demographic balance, reducing model bias toward majority groups at the cost of potential overfitting (oversampling) or information loss (undersampling).