Translation Equivariance
What You'll Learn
You'll learn why convolutional layers naturally detect features regardless of where they appear in an image—a property called translation equivariance.
The Core Idea
Translation equivariance means that when you shift the input, the output shifts by the same amount. In simpler terms: if you move a feature (like an edge or pattern) to a different location in your image, the convolutional layer's response moves to the corresponding location in its output.
This happens because of weight sharing—the same kernel slides across the entire input image, applying identical operations everywhere. Unlike a fully connected layer where each position would have different weights, a convolutional kernel uses the same parameters at every location.
Why This Matters
Imagine teaching a computer to recognize a cat. With translation equivariance:
- A cat in the top-left corner triggers the same feature detectors as a cat in the bottom-right
- The kernel that detects whiskers works identically whether the whiskers are at position (10, 20) or (100, 200)
- You don't need separate weights for "cat in top-left" versus "cat in center"
This dramatically reduces the parameters needed and makes the network naturally robust to position changes—without any extra training data showing the object in every possible location.
The Mathematical Intuition
If you shift your input by 5 pixels to the right, the activation map (output) also shifts 5 pixels to the right. The pattern is preserved; only its location changes. This is equivariance: Conv(shift(x)) = shift(Conv(x)).
Key Takeaway: Translation equivariance, arising from weight sharing in convolutional layers, allows CNNs to detect features anywhere in the input without learning position-specific parameters—making them efficient and naturally position-invariant.