Course contentsShow
← Computer Science
Lesson 2807 of 2,87262. Machine Learning FundamentalsFree lesson

Signs of Overfitting

Identifying overfitting through high training accuracy but poor test performance and excessive model complexity.

Signs of Overfitting

You'll learn how to detect overfitting by recognizing telltale patterns in model performance and complexity.

The Core Warning Signs

Overfitting reveals itself through a characteristic performance gap: your model looks brilliant on training data but stumbles badly on new, unseen test data. Think of it like memorizing exam answers without understanding the concepts—you ace practice tests but fail when questions are slightly reworded.

Primary Indicators

1. Training-Test Performance Divergence
The most obvious red flag is when training accuracy climbs very high (often 95%+) while test accuracy plateaus or even decreases. This gap widens as training continues, showing your model is learning the noise and quirks of your specific training examples rather than generalizable patterns.

2. Excessive Model Complexity
If your model has far more parameters than training examples, or if it has unnecessary layers of sophistication, it has the capacity to memorize rather than learn. A polynomial with 100 terms fitting 50 data points, or a deep neural network trained on a tiny dataset, are classic examples.

Behavioral Clues

During training, watch for these patterns:

  • Validation loss starts increasing while training loss keeps decreasing
  • Perfect or near-perfect training scores (100% accuracy, zero loss)
  • Model performance is extremely sensitive to small changes in training data
  • Complex decision boundaries that twist around individual training points rather than following smooth, general trends

Connecting to Your Tools

Since you've learned about the validation set and cross-validation, you can detect overfitting by monitoring performance across these splits. The bias-variance tradeoff you studied explains why this happens: overfitting means high variance—your model varies wildly with different data samples.

Key Takeaway: Overfitting shows itself through a large gap between training and test performance, combined with unnecessarily complex models that memorize rather than generalize.