EfficientNet Architecture and MBConv Blocks
What You'll Learn
You'll understand how EfficientNet uses mobile inverted bottleneck convolution (MBConv) blocks with squeeze-excitation to achieve state-of-the-art efficiency.
The Core Architecture
EfficientNet builds on the innovations of MobileNetV2 by using MBConv blocks as its fundamental building unit. Think of MBConv as an assembly line that expands, processes, and compresses information efficiently.
The MBConv Block Structure
An MBConv block follows this sequence:
- Expansion: Uses 1×1 convolutions to expand channels (typically 6x)
- Depthwise Processing: Applies depthwise separable convolutions on expanded channels
- Squeeze-and-Excitation: Adds channel attention to recalibrate feature importance
- Projection: Uses 1×1 convolutions to project back to fewer channels
- Skip Connection: Adds residual connection when input/output dimensions match
Squeeze-and-Excitation Enhancement
The squeeze-excitation (SE) mechanism is like a smart spotlight that learns which channels matter most:
- Squeeze: Global average pooling condenses spatial information per channel
- Excitation: Two fully connected layers learn channel importance weights
- Recalibration: Multiply features by learned weights to emphasize important channels
This attention mechanism costs minimal computation but significantly improves accuracy.
Why This Design Works
The inverted bottleneck (narrow→wide→narrow) processes information in high dimensions where depthwise convolutions are most effective, while keeping input/output compact for efficient skip connections. Combined with SE blocks, MBConv achieves excellent accuracy-efficiency trade-offs.
Key Takeaway: EfficientNet's MBConv blocks combine inverted residual structure from MobileNetV2 with squeeze-excitation attention, creating an efficient building block that balances accuracy and computational cost through expansion, depthwise processing, channel attention, and compression.