Course contentsShow
← Machine Learning and Deep Learning
Lesson 1707 of 3,53837. LLM Fine-Tuning: Full and Parameter-EfficientFree lesson

Catastrophic Forgetting in Fine-Tuning

How aggressive fine-tuning can overwrite pretrained knowledge and strategies to preserve general capabilities.

Catastrophic Forgetting in Fine-Tuning

You'll learn how aggressive fine-tuning can erase a model's pretrained knowledge and strategies to preserve general capabilities.

The Problem: Overwriting What the Model Knows

When you fine-tune an LLM on a specific task or narrow dataset, you update all parameters through backpropagation. If your fine-tuning data is limited or highly specialized, the model can catastrophically forget the broad knowledge it learned during pretraining. Think of it like a polyglot who moves to a single country and gradually forgets all languages except the local one.

After aggressive fine-tuning on medical Q&A, for example, your model might excel at diagnosing diseases but suddenly struggle with basic arithmetic or general conversation—capabilities it had before.

Why It Happens

During full fine-tuning, gradients computed from your narrow dataset push all layers' weights away from their pretrained values. With a high learning rate or many epochs, these updates can completely overwrite the rich, general-purpose representations the model learned from its massive pretraining corpus.

Strategies to Preserve General Capabilities

  1. Lower learning rates: Use rates 10-100× smaller than pretraining (e.g., 1e-5 instead of 1e-4) to make gentler updates
  2. Fewer training epochs: Stop early before overfitting to your specific task
  3. Larger, more diverse fine-tuning datasets: Include varied examples that maintain general knowledge
  4. Regularization techniques: Add constraints that keep weights close to pretrained values
  5. Mix in pretraining data: Interleave original pretraining samples with task-specific data during fine-tuning

Key Takeaway: Catastrophic forgetting occurs when fine-tuning too aggressively overwrites pretrained knowledge; mitigate it with conservative learning rates, early stopping, and diverse training data to maintain the model's general capabilities alongside new task-specific skills.