Resampling: Oversampling the Minority Class
What you'll learn: How to balance your dataset by duplicating minority class samples and when this simple technique helps versus hurts your model.
The Core Idea
When you have class imbalance (say, 95% negative cases and 5% positive), your classifier barely sees positive examples during training. Oversampling means creating more copies of the minority class samples so the training set becomes more balanced.
Think of it like studying for an exam: if you only see 1 French vocabulary word for every 19 English words, you'll struggle to learn French. Oversampling is like writing that French word on 19 flashcards so you see both languages equally often.
Random Oversampling Strategy
The simplest approach:
- Identify minority class samples
- Randomly duplicate them (with replacement) until classes are balanced
- Train your model on this augmented dataset
For example, if you have 950 majority samples and 50 minority samples, you'd randomly sample from those 50 (allowing repeats) until you have ~950 minority samples too.
When It Helps vs When It Hurts
Helps when:
- Your minority class has diverse examples that represent the true pattern
- You have a reasonably-sized minority class (at least dozens of examples)
- Combined with regularization to prevent memorization
Causes overfitting when:
- You have very few unique minority samples (e.g., only 10)
- Exact duplicates force the model to memorize specific instances
- Your model is already complex with high capacity
The danger: If you duplicate the same 10 examples 100 times each, your model just memorizes those 10 cases rather than learning generalizable patterns.
Key Takeaway: Random oversampling balances your training data by duplicating minority samples, helping models see underrepresented classes more often—but excessive duplication of few examples leads to overfitting rather than true learning.