Depthwise Separable Convolution
What you'll learn: How to dramatically reduce computation in CNNs by splitting standard convolution into two lightweight steps.
The Core Idea
Depthwise separable convolution factorizes a standard convolution into two separate operations: a depthwise convolution followed by a pointwise convolution. This decomposition achieves nearly the same representational power while using far fewer parameters and computations.
Think of it like packing a suitcase: instead of carefully arranging every item considering all dimensions at once (standard convolution), you first organize items by category in separate compartments (depthwise), then decide how to combine compartments (pointwise).
How It Works
Step 1: Depthwise Convolution
Apply a single filter to each input channel independently. If you have 64 input channels, you use 64 separate filters (one per channel), with no mixing across channels. This is essentially grouped convolution where the number of groups equals the number of channels.
Step 2: Pointwise Convolution
Apply a 1×1 convolution to combine the outputs from the depthwise step. This mixes information across channels without spatial filtering.
Efficiency Gains
For an input with C channels, kernel size k×k, and M output channels:
- Standard convolution:
k × k × C × Mparameters - Depthwise separable:
k × k × C + C × Mparameters
The savings increase dramatically with larger kernel sizes and channel counts—often reducing computation by 8-10× while maintaining comparable accuracy.
Key Takeaway: Depthwise separable convolution factorizes standard convolution into depthwise (spatial filtering per channel) and pointwise (channel mixing) operations, drastically reducing parameters and computation with minimal accuracy loss.