SMOTE: Synthetic Minority Over-sampling
What you'll learn: How SMOTE creates synthetic minority samples through interpolation to improve classifier performance on imbalanced datasets.
The Core Problem with Simple Oversampling
In the previous lesson, you learned about oversampling—simply duplicating minority class examples. While this balances class counts, it doesn't add new information. The model just sees the same examples repeatedly, which can lead to overfitting.
How SMOTE Works
SMOTE (Synthetic Minority Over-sampling Technique) generates new synthetic examples instead of copying existing ones. Here's the mechanism:
- Select a minority sample from your training data
- Find its k nearest neighbors (typically k=5) from the same minority class
- Choose one neighbor randomly
- Interpolate between the original sample and the chosen neighbor
The interpolation creates a new point along the line connecting the two samples. Think of it like blending two paint colors—you get a new shade that shares characteristics of both but isn't identical to either.
Key Hyperparameters
k_neighbors: How many nearest neighbors to consider (default: 5). Smaller k creates samples closer to existing points; larger k increases diversity.sampling_strategy: How much to oversample (e.g., balance classes completely, or reach a specific ratio).
Why Interpolation Beats Duplication
Synthetic examples generalize better because they:
- Expand the decision boundary around minority regions
- Add variation without exact repetition
- Help the model learn the minority class "space" rather than memorizing specific instances
Unlike duplication, SMOTE forces your classifier to consider a broader region of the feature space as belonging to the minority class.
Key Takeaway: SMOTE improves upon simple oversampling by interpolating between minority class neighbors to create synthetic examples that help models generalize better without overfitting to duplicated data.