Course contentsShow
← Machine Learning and Deep Learning
Lesson 472 of 3,53811. Model Evaluation MetricsFree lesson

Adjusted R² for Model Comparison

Learn how adjusted R² penalizes model complexity and why it's essential for comparing models with different numbers of features.

Adjusted R² for Model Comparison

What you'll learn: How adjusted R² corrects for model complexity to fairly compare regression models with different numbers of features.

The Problem with R²

You've learned that R² score measures how well your model explains variance in the target variable. But R² has a quirk: it always increases (or stays the same) when you add more features—even if those features are completely useless! This makes it unreliable when comparing models with different numbers of predictors.

What Adjusted R² Does

Adjusted R² applies a penalty for each additional feature you add to your model. Think of it like this: if you're judging two students' essays, one who wrote 500 words and another who wrote 2000 words, adjusted R² is like grading "quality per word" instead of just total quality. The longer essay doesn't automatically win.

The formula adjusts R² based on:

  • Number of features (p): More features = bigger penalty
  • Sample size (n): Larger datasets reduce the penalty per feature

Why It Matters for Comparison

Imagine comparing two models:

  • Model A: 3 features, R² = 0.85
  • Model B: 20 features, R² = 0.87

Model B looks better, but is it really? Adjusted R² might reveal that Model A is actually superior because Model B's slight improvement doesn't justify adding 17 extra features. The simpler model is more efficient.

When to Use It

Use adjusted R² when:

  • Comparing models with different numbers of features
  • Deciding whether to keep or remove predictors
  • Avoiding overfitting by penalizing unnecessary complexity

Key Takeaway: Adjusted R² penalizes model complexity, making it the right metric for comparing regression models with different feature counts—it rewards simplicity and punishes bloat.