High Bias Diagnosis
What you'll learn: How to recognize when your model is too simple to capture the patterns in your data.
What Is High Bias?
High bias (also called underfitting) occurs when your model is too simple to learn the underlying patterns in your data. Think of it like trying to fit a straight line through data that naturally curves — no matter how you position that line, it won't capture the true relationship.
When you plot learning curves (which you learned in the previous lesson), high bias reveals itself through a telltale signature: both your training error AND validation error remain high, even as you add more training data.
The Diagnostic Pattern
Here's what high bias looks like:
- Training error is high — your model struggles even on the data it's supposed to learn from
- Validation error is high — and it's close to the training error (small gap between them)
- Both errors plateau — adding more data doesn't help much because the model lacks the capacity to learn
Real-World Analogy
Imagine trying to memorize Shakespeare using only three-word phrases. No matter how many times you read the text (more training data), you'll never capture the full richness because your "model" (three-word limit) is fundamentally too simple for the task.
Why This Matters
Diagnosing high bias tells you that your model needs more complexity: more features, a deeper network, removing regularization, or choosing a more expressive model architecture. It's not a data quantity problem — it's a model capacity problem.
Key Takeaway: High bias is diagnosed when both training and validation errors are high and converge together, signaling that your model is too simple to capture the underlying patterns in your data.