Course contentsShow
← Machine Learning and Deep Learning
Lesson 818 of 3,53819. PyTorch FundamentalsFree lesson

Collate Functions: Custom Batch Creation

Master custom collate functions to handle variable-length sequences, padding, and complex batch structures.

Collate Functions: Custom Batch Creation

What You'll Learn

You'll learn how to write custom collate functions that transform lists of dataset samples into properly structured batches, handling tricky cases like variable-length sequences and complex data formats.

The Problem DataLoader Solves (And Creates)

By default, PyTorch's DataLoader stacks your samples into batches using a simple default collate function. This works perfectly when all samples have identical shapes—but real-world data often doesn't cooperate. Text sequences have different lengths, images might vary in size, or you might have complex nested structures that need special handling.

What Is a Collate Function?

A collate function is a custom function you pass to DataLoader that controls exactly how individual samples get combined into a batch. It receives a list of samples (whatever your __getitem__ returns) and must return a single batch structure.

Think of it like organizing groceries: the default method just stacks boxes uniformly, but a custom collate function knows to carefully arrange fragile items, group frozen goods, and handle oddly-shaped packages.

Common Use Cases

Variable-length sequences: Pad text or time-series data to the same length within each batch, creating a tensor plus a mask indicating real vs padded values.

Complex structures: When samples contain multiple elements (image, caption, metadata), collate functions organize them into separate batch tensors or dictionaries.

Dynamic padding: Instead of padding all sequences to a global maximum, pad only to the longest sequence in that specific batch, saving memory and computation.

Implementation Pattern

def custom_collate_fn(batch):
    # batch is a list of samples from __getitem__
    # Return properly structured batch tensors
    pass

loader = DataLoader(dataset, batch_size=32, collate_fn=custom_collate_fn)

Key Takeaway: Custom collate functions give you precise control over batch creation, letting you handle variable-length data, apply padding strategies, and structure complex batches exactly as your model requires.