Course contentsShow
← AI Engineering
Lesson 69 of 1,8862. Working with Pre-trained ModelsFree lesson

Model Pruning Fundamentals

Introduction to structured and unstructured pruning to reduce model size and inference time.

Model Pruning Fundamentals

What you'll learn: How to reduce model size and speed up inference by removing unnecessary weights without sacrificing much accuracy.

The Core Idea

Model pruning is like decluttering a house—you remove items you don't use while keeping everything essential. Neural networks often contain many weights (parameters) that contribute little to the final predictions. Pruning identifies and removes these less important weights, creating a smaller, faster model.

Think of it like editing a essay: you can often cut 20% of the words while preserving 95%+ of the meaning.

Two Main Pruning Approaches

Unstructured Pruning

This removes individual weights scattered throughout the model based on their magnitude (size). If a weight is close to zero, it's probably not doing much, so we zero it out.

  • Pros: Can achieve high sparsity (many zeros)
  • Cons: Irregular patterns make hardware acceleration tricky—sparse matrices don't always run faster on standard GPUs

Structured Pruning

This removes entire structures like neurons, attention heads, or layers. Instead of random individual weights, you eliminate whole rows, columns, or blocks.

  • Pros: Creates truly smaller models that run faster on any hardware
  • Cons: Less granular—might remove slightly more important weights

Why It Matters

After pruning with attention mechanism optimization (#68) and memory management (#65), you can often:

  • Reduce model size by 30-50%
  • Speed up inference by 1.5-2x
  • Lower memory usage significantly

The trade-off is typically <3% accuracy loss when done carefully.

Key Takeaway: Pruning removes unnecessary model weights—either individual parameters (unstructured) or entire structural components (structured)—to create smaller, faster models with minimal accuracy loss.