Course contentsShow
← Computer Science
Lesson 2808 of 2,87262. Machine Learning FundamentalsFree lesson

The Bias-Variance Tradeoff

Understanding the tradeoff between bias (underfitting) and variance (overfitting) in model performance.

The Bias-Variance Tradeoff

You'll learn: How to balance a model's ability to capture patterns without memorizing noise, by understanding the opposing forces of bias and variance.

What Is This Tradeoff?

Every machine learning model faces a fundamental tension: bias (underfitting) pulls in one direction, variance (overfitting) pulls in the other. Understanding this tradeoff helps you find the sweet spot where your model generalizes best to new data.

The Two Forces

Bias measures how much your model's predictions systematically miss the true patterns. High bias means your model is too simple—it can't capture the underlying relationships in your data. Think of trying to fit a straight line to data that curves: no matter how you adjust that line, you'll consistently miss the mark.

Variance measures how much your model's predictions change when trained on different subsets of data. High variance means your model is too complex—it's memorizing noise and peculiarities of your training set instead of learning general patterns. Like a student who memorizes specific test questions but can't apply concepts to new problems.

The Balancing Act

As you increase model complexity:

  • Bias decreases (the model can fit more intricate patterns)
  • Variance increases (the model becomes sensitive to training data quirks)

As you decrease model complexity:

  • Bias increases (the model can't capture true relationships)
  • Variance decreases (predictions become more stable)

Total error comes from both sources plus irreducible noise. Your goal is finding the model complexity that minimizes their sum.

Practical Connection

Remember from your earlier lessons: overfitting shows low training error but high test error (high variance), while underfitting shows high error on both (high bias).

Key Takeaway: The bias-variance tradeoff is central to model selection—you must balance simplicity and complexity to achieve optimal generalization performance.