Why R-Squared Alone Is Misleading
What You'll Learn
You'll discover why R-squared can give you a false sense of model improvement when you add more predictors to your regression model.
The R-Squared Trap
By now, you know that R-squared measures how much variance your model explains. Here's the catch: R-squared can only increase (or stay the same) when you add more predictors—it will never decrease. This happens even if the new predictors are completely useless or just random noise.
Think of it like padding your résumé with irrelevant skills. Sure, your résumé gets longer and looks more impressive at first glance, but are you actually more qualified? Not necessarily.
Why This Happens
When you add a new predictor, the least squares algorithm finds some relationship between it and the residuals, even if it's purely by chance. The algorithm doesn't know whether that relationship is real or meaningless—it just fits the best line it can find. This always captures at least a tiny bit more variance, nudging R-squared upward.
The Problem in Practice
Imagine comparing two models:
- Model A: Predicts house price using square footage (R² = 0.65)
- Model B: Predicts house price using square footage + owner's favorite color (R² = 0.652)
Model B has a higher R-squared, but the "favorite color" variable adds no real predictive value—it's just fitting noise. R-squared alone would mislead you into thinking Model B is better.
What This Means for You
When building multiple regression models, you need tools that penalize complexity—that is, they account for how many predictors you're using, not just how well the model fits.
Key Takeaway: R-squared alone is misleading in multiple regression because it always increases with more predictors, rewarding overly complex models that may just be fitting noise rather than true patterns.