Course contentsShow
← Machine Learning and Deep Learning
Lesson 2665 of 3,53858. Model Compression: Pruning and DistillationFree lesson

What Is Neural Network Pruning?

Understanding pruning as removing unnecessary weights or neurons to reduce model size while preserving performance.

What Is Neural Network Pruning?

You'll learn: How pruning removes unnecessary parts of a neural network to compress models without sacrificing much accuracy.

The Core Idea

Neural network pruning is the process of identifying and removing weights, neurons, or entire structural components that contribute little to the model's predictive performance. Think of it like pruning a tree—you cut away dead or unnecessary branches so the tree remains healthy while using fewer resources.

After training, many neural networks contain redundant parameters. Some weights are near zero and barely influence predictions. Some neurons fire weakly or correlate heavily with others. Pruning eliminates these elements, creating a smaller, faster model that often performs nearly as well as the original.

Why Prune?

Modern deep learning models can have millions or billions of parameters. Pruning addresses three key challenges:

  • Model size reduction: Fewer parameters mean smaller files for deployment on mobile devices or edge hardware
  • Faster inference: Less computation during forward passes speeds up predictions
  • Energy efficiency: Fewer operations consume less power

The Pruning Process

The typical workflow involves:

  1. Train a large, overparameterized model to full accuracy
  2. Identify which weights or neurons to remove (based on magnitude, gradient sensitivity, or learned importance scores)
  3. Remove those elements by setting weights to zero or physically deleting connections
  4. Fine-tune the pruned model to recover any lost accuracy

Unlike quantization (which reduces numerical precision), pruning changes the network's structure itself—eliminating connections or entire nodes.

Key Takeaway: Pruning compresses neural networks by removing weights or neurons that have minimal impact, reducing model size and inference cost while maintaining most of the original performance.