Course contentsShow
← Machine Learning and Deep Learning
Lesson 918 of 3,53821. CNN ArchitecturesFree lesson

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Introducing inverted residual blocks with expansion layers and linear bottlenecks to preserve information flow.

MobileNetV2: Inverted Residuals and Linear Bottlenecks

What You'll Learn

You'll discover how MobileNetV2 improves upon MobileNetV1 by reversing the traditional bottleneck structure and using linear activations to preserve critical information.

The Inverted Residual Block

MobileNetV1 used depthwise separable convolutions, but MobileNetV2 introduces a clever twist: inverted residual blocks.

Traditional bottleneck blocks (like in ResNet) follow a "wide → narrow → wide" pattern: they compress features, process them, then expand again. MobileNetV2 does the opposite: "narrow → wide → narrow."

How It Works

  1. Expansion Layer: Start with low-dimensional input and expand it using a 1×1 convolution (typically 6× expansion)
  2. Depthwise Convolution: Apply efficient depthwise separable convolution in this high-dimensional space
  3. Linear Bottleneck: Compress back down with a 1×1 convolution, but without ReLU activation

Think of it like inflating a balloon to do work in a larger space, then carefully deflating it to preserve the results.

Why Linear Bottlenecks?

The "linear bottleneck" means the final projection layer has no activation function. This is crucial because ReLU destroys negative values. When compressing information into a low-dimensional space, ReLU can erase important features permanently.

By keeping the bottleneck linear, MobileNetV2 preserves the manifold of useful features, allowing information to flow through the network more effectively—similar to ResNet's skip connections, but optimized for efficiency.

The Complete Picture

The inverted residual block also includes a skip connection that adds the input directly to the output (when dimensions match), enabling gradient flow like ResNet while maintaining extreme computational efficiency.

Key Takeaway: MobileNetV2's inverted residuals expand features before processing them efficiently, then use linear bottlenecks to compress results without destroying information—achieving better accuracy than MobileNetV1 with similar computational cost.