Dilated Convolution
What you'll learn: How dilated convolution expands your network's receptive field without adding extra parameters or computation.
The Core Idea
Dilated convolution (also called atrous convolution) spaces out the kernel elements by inserting gaps between them. Instead of applying your filter to adjacent pixels, you skip pixels according to a dilation rate. A dilation rate of 1 gives you standard convolution. A rate of 2 means you skip one pixel between each kernel element. A rate of 3 skips two pixels, and so on.
Why This Matters
Recall from earlier lessons that a 3×3 filter only sees 3×3 pixels at once—that's your receptive field. To see more context, you'd traditionally stack more layers or use larger kernels (like 5×5 or 7×7), both of which increase parameters.
Dilated convolution offers a third option: keep the same kernel size and parameter count, but spread the kernel out to cover a wider area. A 3×3 kernel with dilation rate 2 actually covers a 5×5 pixel area (but only uses 9 parameters). With rate 3, it covers 7×7 pixels—still just 9 parameters!
The Analogy
Think of a standard convolution as reading every word in a sentence. Dilated convolution is like reading every other word (rate 2) or every third word (rate 3)—you're sampling a longer sentence with the same number of words read, capturing broader context efficiently.
This makes dilated convolutions especially powerful in tasks requiring large receptive fields without deep networks: semantic segmentation, audio generation, and dense prediction tasks.
Key Takeaway: Dilated convolution exponentially expands receptive field size while maintaining the same parameter count and computational cost as standard convolution, by introducing spacing between kernel elements.