What Are Residuals?
You'll learn: How residuals measure the "mistakes" your regression model makes and why they're essential for evaluating model quality.
The Core Idea
A residual is simply the difference between what you actually observed in your data and what your regression model predicted. Think of it as the prediction error for each individual data point.
If your model predicts a value of ŷ for a data point, but the actual observed value is y, then:
Residual = Observed − Predicted = y − ŷ
Why Residuals Matter
Imagine you're predicting house prices based on square footage. Your regression line gives you predicted prices, but the actual sale prices rarely land exactly on that line. Each vertical distance from a data point to the regression line is a residual.
Residuals tell you:
- How well your model fits: Small residuals mean your predictions are close to reality; large residuals suggest poor fit
- Whether assumptions hold: Patterns in residuals can reveal problems (non-linearity, unequal variance, outliers)
- Where your model struggles: Which observations are hardest to predict
Positive vs Negative Residuals
- Positive residual: Your model under-predicted (actual value was higher than predicted)
- Negative residual: Your model over-predicted (actual value was lower than predicted)
- Zero residual: Perfect prediction (the point lies exactly on the regression line)
A good model has residuals that are small and randomly scattered around zero, with no systematic patterns.
Key Takeaway: Residuals are the prediction errors (observed minus predicted values) that reveal how well your model fits the data and help diagnose potential problems with your regression assumptions.