Limitations of R-Squared
What You'll Learn
You'll discover why R-squared alone can't tell you if your model is truly good, causal, or correctly specified.
The Problem with R-Squared
R-squared tells you what proportion of variance your model explains—but it has serious blind spots that can mislead beginners.
R-Squared Always Increases with More Predictors
Here's a critical flaw: R-squared never decreases when you add variables. Even if you add completely random, meaningless predictors to your model, R-squared will go up (or stay the same). It's like claiming you're getting "better" at predictions just by throwing in more ingredients, even if some are irrelevant.
Think of it like a lock with 100 keyholes—trying more keys will eventually open it by chance, but that doesn't mean each key was actually useful.
R-Squared Doesn't Indicate Causation
You might have an R-squared of 0.90, suggesting your X variable "explains" 90% of variation in Y. But this says nothing about whether X causes Y. Remember your earlier lessons on correlation vs causation—the same warning applies here. High R-squared with ice cream sales and drowning deaths doesn't mean ice cream causes drowning; both are driven by summer weather (a confounding variable).
R-Squared Doesn't Validate Your Model Specification
A high R-squared doesn't mean:
- Your relationship is actually linear
- You've included the right variables
- Your residuals meet assumptions
- Your model will predict well on new data
You could have a terrible model with violated assumptions but still achieve high R-squared.
Key Takeaway
R-squared is a useful but incomplete metric: it automatically increases with added predictors, cannot establish causation, and doesn't confirm you've built the correct model—always combine it with residual diagnostics, domain knowledge, and causal reasoning.