Handling Imbalanced Data in DataLoaders
What You'll Learn
You'll learn how to use WeightedRandomSampler and batch sampling strategies to ensure your model trains fairly on datasets where some classes appear far more often than others.
The Problem: When Your Data Isn't Balanced
Imagine training a fraud detection model where 99% of transactions are legitimate and only 1% are fraudulent. If you randomly sample batches, your model might see hundreds of normal transactions before encountering a single fraud case—leading it to simply ignore the rare class entirely.
Imbalanced data means some classes have many more examples than others. Without intervention, your DataLoader will naturally show the model mostly majority-class examples, causing poor performance on minority classes.
The Solution: Weighted Sampling
WeightedRandomSampler lets you assign sampling probabilities to each example. Rare classes get higher weights, making them appear more frequently during training—like giving the fraud cases "extra tickets" in a lottery.
The basic strategy:
- Calculate the weight for each sample (inversely proportional to class frequency)
- Pass these weights to
WeightedRandomSampler - Use the sampler in your DataLoader
This rebalances your batches so the model sees a more representative mix of classes, even though the underlying dataset remains imbalanced. You're not changing the data itself—just how often each example gets selected during training.
Think of it like a teacher calling on quiet students more often to ensure everyone participates equally, even if extroverts naturally dominate discussions.
When to Use It
Use weighted sampling when:
- Class distribution is skewed (e.g., 90:10 or worse)
- Minority class performance matters significantly
- You want balanced learning across all classes
Key Takeaway: WeightedRandomSampler rebalances training by making rare examples appear more frequently in batches, preventing your model from ignoring minority classes without modifying your dataset.