Course contentsShow
← Machine Learning and Deep Learning
Lesson 142 of 3,5382. Machine Learning Mindset and Core ConceptsFree lesson

The Bias-Variance Tradeoff

Understanding the fundamental tradeoff between underfitting and overfitting in ML models.

The Bias-Variance Tradeoff

What you'll learn: How the two main sources of prediction error—bias and variance—compete in machine learning models, and why balancing them is essential for good generalization.

The Core Idea

Every machine learning model makes prediction errors. These errors come from two fundamental sources:

  • Bias measures how far off your model's average predictions are from the true values. High bias means your model is too simple—it underfits and misses important patterns in the data.
  • Variance measures how much your model's predictions change when trained on different datasets. High variance means your model is too complex—it overfits and memorizes noise instead of learning true patterns.

The Tradeoff

Here's the catch: reducing one often increases the other.

Think of it like archery. Bias is like consistently aiming too far left—your arrows cluster together but miss the bullseye. Variance is like having arrows scattered all over the target—sometimes you hit, sometimes you don't. The ideal archer (model) aims accurately (low bias) and hits consistently in the same spot (low variance).

When you make your model more complex:

  • Bias decreases (it can capture more patterns)
  • Variance increases (it becomes sensitive to training data quirks)

When you simplify your model:

  • Bias increases (it misses subtle patterns)
  • Variance decreases (it's more stable across datasets)

Your goal is to find the sweet spot where total error is minimized—not too simple, not too complex.

Connection to Earlier Concepts

Remember baseline models and train-validation-test splits? Baseline models are high-bias (simple), while monitoring validation performance helps you detect when variance starts growing.

Key Takeaway: The bias-variance tradeoff explains why model complexity must be carefully tuned—you need enough capacity to learn true patterns (reduce bias) without memorizing noise (increase variance).