Residual Analysis and Diagnostic Plots
What you'll learn: How to use residuals (prediction errors) to diagnose whether your regression model is trustworthy or hiding problems.
What Are Residuals?
A residual is simply the difference between what your model predicted and the actual value: residual = actual - predicted. Think of residuals as your model's "mistakes." If your model is working properly, these mistakes should look random—like static on a radio. But if patterns appear in the residuals, your model has systematic problems.
Why Residual Analysis Matters
After calculating metrics like MAE, MSE, or R², you know how much your model errs, but not how it errs. Residual plots reveal hidden issues:
- Non-linearity: If residuals form a curve, your linear model is trying to fit a curved relationship
- Heteroscedasticity: If the spread of residuals increases/decreases along predictions, your model's confidence varies unreliably (violates constant variance assumption)
- Outliers: Extreme residuals point to unusual data points that may distort your model
Common Diagnostic Plots
- Residuals vs Predicted Values: Should show random scatter around zero with constant spread. Patterns indicate problems.
- Residuals vs Features: Helps identify which features cause issues.
- Q-Q Plot: Compares residual distribution to normal distribution. Points should follow a straight line if residuals are normally distributed (an assumption for many statistical tests).
- Scale-Location Plot: Shows if residual spread changes with predicted values.
Think of residual analysis like a medical diagnostic: your metrics tell you the patient has a fever, but diagnostic plots reveal whether it's a cold or something serious.
Key Takeaway: Residual plots transform invisible model assumptions into visible patterns—random scatter means your model assumptions hold; patterns mean you need to rethink your approach.