Course contentsShow
← Machine Learning and Deep Learning
Lesson 204 of 3,5383. Linear RegressionFree lesson

Multicollinearity and Its Effects

Detecting and understanding when features are correlated and how it destabilizes coefficient estimates.

Multicollinearity and Its Effects

You'll learn how correlated features can make your linear regression coefficients unreliable and unstable.

What is Multicollinearity?

Multicollinearity occurs when two or more features in your dataset are highly correlated with each other. Imagine you're predicting house prices using both "square footage" and "number of rooms" as features. These variables tend to move together—bigger houses usually have more rooms. When features are strongly correlated like this, it creates a problem for multiple linear regression.

Why It Matters

When solving the Normal Equation, multicollinearity makes the feature matrix nearly singular (close to non-invertible). This means small changes in your training data can cause huge swings in the estimated coefficients.

Think of it like this: if you're trying to determine whether square footage or number of rooms drives price, but they always change together, how can you tell which one matters? The model gets confused and may assign arbitrary weights that happen to work mathematically but don't reflect reality.

The Effects

  1. Unstable coefficients: Small data changes cause large coefficient changes
  2. Inflated standard errors: Coefficients become statistically unreliable
  3. Hard to interpret: You can't trust which features are "important"
  4. Predictions still work: Interestingly, predictions may remain accurate even though individual coefficients are unreliable

You can detect multicollinearity by checking correlation matrices between features or computing the Variance Inflation Factor (VIF) for each predictor.

Key Takeaway: Multicollinearity doesn't break predictions, but it makes coefficient estimates unstable and uninterpretable—defeating one of linear regression's main advantages: understanding feature importance.