Reduction Operations
What you'll learn: How to collapse tensors along one or more dimensions using aggregation functions like sum, mean, max, and min.
The Core Idea
Reduction operations transform tensors by computing aggregate values along specified dimensions. Think of it like summarizing a spreadsheet: you might sum all values in a column, find the maximum across rows, or calculate the average of an entire table. In PyTorch, these operations "reduce" or collapse dimensions while preserving important statistical information.
Common Reduction Functions
PyTorch provides several reduction operations:
.sum()— adds all values.mean()— computes the average.max()— finds the largest value.min()— finds the smallest value
Dimension Control
You can reduce:
- All dimensions (entire tensor → single value)
- Specific dimension(s) using the
dimparameter - With
keepdim=Trueto preserve the reduced dimension as size 1
import torch
tensor = torch.tensor([[1.0, 2.0, 3.0],
[4.0, 5.0, 6.0]])
# Reduce everything to single value
total = tensor.sum() # 21.0
# Sum along dimension 0 (collapse rows)
col_sums = tensor.sum(dim=0) # [5.0, 7.0, 9.0]
# Mean along dimension 1 (collapse columns)
row_means = tensor.mean(dim=1) # [2.0, 5.0]
# Keep dimension for broadcasting
row_means_kept = tensor.mean(dim=1, keepdim=True) # [[2.0], [5.0]]
The keepdim parameter is crucial when you need to broadcast results back to the original shape or perform further operations that require dimension alignment.
Key Takeaway: Reduction operations aggregate tensor values along specified dimensions using .sum(), .mean(), .max(), and .min(), with keepdim=True preserving dimension structure for subsequent operations.