Course contentsShow
← Machine Learning and Deep Learning
Lesson 937 of 3,53822. Computer Vision TasksFree lesson

Layer Freezing Strategies

Deciding which layers to freeze or train based on dataset size and similarity to pretraining data.

Layer Freezing Strategies

What you'll learn: How to decide which layers of a pretrained network to freeze or train based on your dataset's size and similarity to the original training data.

The Core Idea

When fine-tuning a pretrained CNN, you don't always want to retrain every layer. Layer freezing means locking certain layers' weights so they don't update during training, while allowing others to learn from your new data.

Think of it like renovating a house: if the foundation is solid and matches your needs, you keep it unchanged. But you might repaint rooms or redesign the kitchen. Similarly, early CNN layers (the "foundation") learn general features like edges and textures that work across many datasets. Later layers learn task-specific patterns.

Decision Framework

Your strategy depends on two factors:

Dataset Size

  • Small dataset (<1,000 images per class): Freeze most layers to prevent overfitting. Only train the final classification head or last few layers.
  • Medium dataset: Freeze early layers, fine-tune middle and late layers.
  • Large dataset: Unfreeze more layers or even train the entire network.

Similarity to Pretraining Data

  • High similarity (e.g., ImageNet → cats vs dogs): Early layers already extract perfect features. Freeze most, train only the top.
  • Low similarity (e.g., ImageNet → medical X-rays): More layers need adaptation. Unfreeze deeper into the network, but still freeze the earliest layers which capture universal visual patterns.

Common Patterns

  • Always retrain the classification head (it's task-specific)
  • Freeze gradually from input → deeper layers
  • Use lower learning rates for unfrozen layers to preserve learned features

Key Takeaway: Freeze more layers when you have less data or high similarity to pretraining data; unfreeze more layers when you have abundant data or domain differences require feature adaptation.