Dilated Convolutions for Large Receptive Fields
What you'll learn: How dilated convolutions exponentially expand receptive fields without adding parameters or losing resolution.
The Core Idea
Standard convolutions grow receptive fields linearly—each layer adds a small amount. But what if you need to "see" a much larger area without stacking dozens of layers or losing spatial resolution through pooling?
Dilated convolutions (also called atrous convolutions) insert gaps between kernel elements, allowing the filter to cover a larger spatial area with the same number of parameters. Instead of a 3×3 kernel covering 9 adjacent pixels, a dilated convolution with dilation rate 2 covers a 5×5 area by sampling every other position.
How It Works
Think of it like reading every other word in a sentence—you cover more ground with the same number of words read.
- Dilation rate 1: Standard convolution (no gaps)
- Dilation rate 2: One pixel gap between kernel elements
- Dilation rate 4: Three pixel gaps between elements
A 3×3 kernel with dilation rate d has an effective receptive field of (3-1)×d + 1 in each dimension.
Why This Matters
Dilated convolutions let you exponentially expand receptive fields without:
- Adding parameters (still 3×3 = 9 weights)
- Reducing spatial resolution (no pooling needed)
- Increasing computational cost significantly
This makes them ideal for tasks like semantic segmentation, where you need both large context and fine-grained spatial information.
Key Takeaway: Dilated convolutions expand receptive fields exponentially by inserting gaps in the kernel, achieving wider context without additional parameters or resolution loss.