Course contentsShow
← Machine Learning and Deep Learning
Lesson 2675 of 3,53858. Model Compression: Pruning and DistillationFree lesson

Structured Pruning: Channel Pruning

Removing entire convolutional filters or channels to achieve true speedup on standard hardware.

Structured Pruning: Channel Pruning

What you'll learn: How to remove entire convolutional filters or channels to achieve actual hardware speedup, not just theoretical parameter reduction.

The Core Idea

Unlike unstructured pruning that zeroes out individual weights (creating sparse matrices that require special hardware), channel pruning removes entire convolutional filters or output channels. When you eliminate a channel from layer i, you delete all the corresponding filters in layer i+1 that process that channel. This creates a genuinely smaller, denser network that runs faster on standard GPUs and CPUs.

Why Channel Pruning?

Think of a convolutional layer as a factory with multiple assembly lines (channels), each producing a specific feature map. Unstructured pruning is like randomly removing workers from each line—the lines still exist, just with gaps. Channel pruning shuts down entire assembly lines. The factory becomes physically smaller and more efficient.

For example, pruning 25% of channels in a Conv2D layer with 256 filters reduces it to 192 filters. No special sparse computation needed—standard dense matrix operations just work faster because there's less data to process.

Selection Criteria

Common approaches to decide which channels to prune:

  • L1-norm of filters: Remove channels whose filter weights have the smallest magnitude
  • Average activation magnitude: Prune channels that produce weak feature maps
  • Gradient-based importance: Keep channels with larger gradients during training

After pruning channels, you must fine-tune the network to recover accuracy, as removing entire feature extractors is more disruptive than unstructured pruning.

Key Takeaway: Channel pruning delivers real-world speedups by physically removing entire convolutional filters, creating smaller dense networks that run efficiently on standard hardware without requiring sparse computation support.