Course contentsShow
← Machine Learning and Deep Learning
Lesson 822 of 3,53819. PyTorch FundamentalsFree lesson

Samplers: Controlling Data Access Patterns

Use built-in and custom Samplers to control batch composition, stratification, and weighted sampling.

Samplers: Controlling Data Access Patterns

What You'll Learn

You'll learn how to use PyTorch Samplers to control which examples appear in each batch, enabling techniques like stratified sampling and weighted selection.

The Core Idea

By default, a DataLoader either shuffles your data randomly or processes it sequentially. But what if you need more control? Samplers let you define exactly which indices get selected and in what order.

Think of a sampler as a smart playlist manager for your training data. Just as a music app might play songs by artist, mood, or play count, a sampler determines which data samples appear in each batch based on rules you define.

Why Samplers Matter

Without samplers, you're stuck with uniform random selection. But real-world scenarios often need:

  • Stratified sampling: Ensuring each batch contains examples from all classes
  • Weighted sampling: Oversampling rare classes to balance imbalanced datasets
  • Sequential access: Deterministic ordering for reproducibility
  • Subset sampling: Training on only part of your dataset

Built-in Samplers

PyTorch provides several ready-to-use samplers:

  • SequentialSampler: Returns indices in order (0, 1, 2, ...)
  • RandomSampler: Shuffles indices randomly each epoch
  • SubsetRandomSampler: Randomly samples from a specified subset of indices
  • WeightedRandomSampler: Samples indices with given probabilities—perfect for class imbalance

The DataLoader's shuffle parameter is actually just shorthand for choosing between RandomSampler and SequentialSampler.

Custom Samplers

You can create custom samplers by subclassing torch.utils.data.Sampler and implementing __iter__() and __len__() methods to define your sampling logic.

Key Takeaway: Samplers give you fine-grained control over batch composition by determining which dataset indices get selected, enabling strategies like class balancing and stratification that go beyond simple shuffling.