Course contentsShow
← Machine Learning and Deep Learning
Lesson 394 of 3,5389. Dimensionality ReductionFree lesson

PCA Limitations and Assumptions

When PCA fails: linearity assumption, sensitivity to outliers, and inability to capture nonlinear manifolds.

PCA Limitations and Assumptions

What you'll learn: The key scenarios where Principal Component Analysis breaks down and produces poor results.

When PCA Falls Short

While PCA is powerful for many datasets, it's not a universal solution. Understanding its limitations helps you choose the right tool for the job.

The Linearity Assumption

PCA only captures linear relationships between features. It finds new axes (principal components) that are linear combinations of your original features. Think of it like trying to summarize a spiral staircase using only straight lines—you'll miss the curves entirely.

If your data lies on a nonlinear manifold (like a Swiss roll or horseshoe shape), PCA will fail to preserve the true structure. It might squash points that should be far apart or separate points that should be close.

Sensitivity to Outliers

Because PCA relies on variance and the covariance matrix, outliers have outsized influence. Remember, variance involves squared distances from the mean. A single extreme point can pull the principal components toward itself, distorting the representation of the majority of your data.

Imagine trying to find the "main direction" of a flock of birds, but one rogue bird is flying far off course—PCA might orient toward that outlier rather than the flock.

Scale Dependence

PCA is affected by feature scales (which is why we standardize first). Features with larger variances dominate the principal components, even if they're not more informative—just measured in different units.

Key Takeaway: PCA assumes linear structure, is vulnerable to outliers, and cannot capture curved or nonlinear manifolds in your data. For complex nonlinear patterns, you'll need techniques like kernel PCA or manifold learning methods.