Course contentsShow
← Machine Learning and Deep Learning
Lesson 924 of 3,53821. CNN ArchitecturesFree lesson

SqueezeNet: Fire Modules and Compression

Achieving AlexNet-level accuracy with 50x fewer parameters using squeeze and expand layers in fire modules.

SqueezeNet: Fire Modules and Compression

What you'll learn: How SqueezeNet achieves AlexNet-level accuracy with 50x fewer parameters through clever architectural design.

The Efficiency Challenge

While AlexNet revolutionized deep learning, it used over 60 million parameters. SqueezeNet proved you could achieve similar accuracy with just 1.2 million parameters—a massive 50x reduction—making models deployable on mobile devices and embedded systems.

The Fire Module: Squeeze and Expand

The fire module is SqueezeNet's core innovation, consisting of two layers:

  1. Squeeze layer: Uses 1×1 convolutions to drastically reduce the number of input channels (think of it as compressing information)
  2. Expand layer: Splits into parallel 1×1 and 3×3 convolutions, then concatenates results (reconstructing richer representations)

Why This Works

Analogy: Imagine summarizing a long document (squeeze) then elaborating key points in different styles (expand). You capture essential information efficiently while maintaining expressiveness.

The squeeze layer creates a bottleneck—far fewer channels than the output. For example, if you have 256 input channels, squeeze might reduce to 16 channels, then expand back to 256. This forces the network to learn compressed, efficient representations.

Design Strategies

SqueezeNet employs three key strategies:

  1. Replace 3×3 filters with 1×1 filters wherever possible (9x parameter reduction per filter)
  2. Decrease input channels to 3×3 filters using squeeze layers
  3. Downsample late in the network to maintain large activation maps

These strategies combined with fire modules create an incredibly parameter-efficient architecture without sacrificing accuracy.

Key Takeaway: SqueezeNet demonstrates that aggressive parameter reduction through squeeze-expand fire modules can maintain accuracy while making models 50x smaller—critical for real-world deployment on resource-constrained devices.