Wide Residual Networks (WRN)
What you'll learn: How making residual networks wider (more filters per layer) rather than deeper can improve performance and training efficiency.
The Core Idea
After ResNet proved that networks could go extremely deep (152+ layers), researchers asked: Is depth the only path to better performance? Wide Residual Networks (WRN) challenge this assumption by increasing the width of each residual block instead of stacking more blocks.
Think of it like building strength: you could do 100 pushups with bodyweight (depth), or 50 pushups while carrying a heavy backpack (width). Sometimes the second approach builds more muscle with less time.
Trading Depth for Width
In standard ResNet, each convolutional layer might have 64, 128, or 256 filters. WRN multiplies these numbers by a widening factor (typically 2-10×). For example:
- ResNet-50: A residual block might have [64, 64, 256] filters across its convolutions
- WRN-28-10: Fewer blocks (28 layers total), but each layer has 10× more filters
Why This Works
- Better parallelization: GPUs handle wider layers more efficiently than very deep sequential processing
- Reduced vanishing gradient risk: Fewer layers means shorter gradient paths
- More representational power per layer: More filters can capture richer features at each stage
- Faster training: Fewer sequential operations mean quicker forward/backward passes
The trade-off? More memory usage per layer and more parameters overall—but often with better accuracy per training hour.
Key Takeaway: Wide Residual Networks demonstrate that increasing network width (filters per layer) can be more effective than extreme depth, offering better performance with improved training speed and GPU utilization.