Comparing Models with Adjusted R-Squared
You'll learn how to use adjusted R-squared to fairly compare regression models with different numbers of predictors.
The Problem with Regular R-Squared
You've learned that regular R-squared always increases (or stays the same) when you add more predictors—even if those predictors are useless. This makes it a poor tool for comparing models with different numbers of variables. Model A with 3 predictors and Model B with 10 predictors can't be fairly compared using R-squared alone.
Adjusted R-Squared as a Fair Judge
Adjusted R-squared solves this by penalizing unnecessary complexity. Think of it like judging ice skaters: if two skaters get similar scores, but one did a simpler routine, that skater is more impressive. Similarly, if two models explain similar variation, the simpler one (fewer predictors) is better.
When comparing models:
- Higher adjusted R-squared suggests a better balance of fit and simplicity
- A model with more predictors must justify their inclusion by improving fit enough to offset the penalty
- If adding a predictor barely improves prediction, adjusted R-squared may actually decrease
When to Use It
Use adjusted R-squared when:
- Comparing models with different numbers of predictors
- Deciding whether to include additional variables
- Performing variable selection (though other methods like AIC and BIC exist too)
Example Comparison
Imagine Model 1 (2 predictors): Adjusted R² = 0.72
Model 2 (5 predictors): Adjusted R² = 0.70
Even though Model 2 has more predictors, its lower adjusted R-squared tells you the extra variables aren't pulling their weight. Model 1 is preferable.
Key Takeaway: Adjusted R-squared levels the playing field when comparing models with different predictor counts, rewarding parsimony—simpler models that explain data well without unnecessary complexity.