Course contentsShow
← Machine Learning and Deep Learning
Lesson 1708 of 3,53837. LLM Fine-Tuning: Full and Parameter-EfficientFree lesson

Training Duration and Convergence

Determining appropriate number of epochs, steps, and early stopping criteria for full fine-tuning runs.

Training Duration and Convergence

You'll learn: How to determine the right number of training epochs, steps, and when to stop fine-tuning an LLM before wasting compute or losing performance.

The Challenge of "How Long?"

When fine-tuning an LLM, you need to train long enough for the model to learn your task, but not so long that it overfits or forgets general capabilities (catastrophic forgetting). Unlike pretraining with massive datasets, fine-tuning typically uses smaller, focused datasets where convergence happens much faster.

Key Training Duration Concepts

Epochs vs Steps

An epoch is one complete pass through your training dataset. Steps are individual optimizer updates (batches processed). With a 10,000-example dataset and batch size 8, one epoch = 1,250 steps.

Monitoring Convergence

Track your validation loss curve. Initially it drops sharply, then plateaus. If validation loss starts rising while training loss keeps falling, you're overfitting. Most LLM fine-tuning runs converge in 1-5 epochs (rarely more than 10).

Early Stopping

Set a patience threshold: if validation loss doesn't improve for N consecutive evaluations, stop training. This prevents wasting compute and catches the model at its best generalization point. Save checkpoints periodically so you can roll back if needed.

Practical Heuristics

  • Small datasets (<10K examples): 3-5 epochs often sufficient
  • Large datasets (>100K): May need only 1-2 epochs
  • High learning rates: Converge faster but risk instability
  • Monitor both loss and task-specific metrics: Sometimes loss plateaus but accuracy still improves

Think of fine-tuning like seasoning a dish—you add gradually, taste frequently, and stop when it's just right. Too little and it's bland; too much and you've ruined the original flavor.

Key Takeaway: Fine-tuning duration is dataset-dependent; use validation metrics and early stopping to catch the optimal point before overfitting or catastrophic forgetting occurs.