Residuals and Prediction Errors
What you'll learn: How to measure the gap between what your model predicts and reality—the foundation of model accuracy assessment.
The Core Idea
When your linear regression model makes predictions, it won't be perfect. For each data point, there's a difference between what the model predicted and the actual true value. This difference is called a residual (or prediction error).
Think of it like a weather forecast: if the meteorologist predicts 75°F but it's actually 70°F, the residual is 5°F. The smaller these errors, the better your model performs.
Mathematical Definition
For a single data point:
Residual = Actual Value - Predicted Value
Or in notation: e = y - ŷ (where y is actual, ŷ is predicted)
If you have multiple data points, you'll have multiple residuals—one for each prediction. These residuals tell you where and by how much your model is wrong.
Why Residuals Matter
Residuals are crucial because they:
- Quantify model accuracy: Large residuals mean poor predictions
- Guide optimization: Most training algorithms try to minimize residuals
- Reveal patterns: Systematic residuals indicate your model is missing something important
For example, if all your residuals are positive, your model consistently underestimates. If they're randomly scattered near zero, your model is doing well.
Positive vs Negative Residuals
- Positive residual: Model underestimated (predicted too low)
- Negative residual: Model overestimated (predicted too high)
- Zero residual: Perfect prediction (rare in practice!)
Key Takeaway: Residuals measure the gap between predictions and reality—they're your model's report card, showing exactly where it succeeds and fails.