Course contentsShow
← Machine Learning and Deep Learning
Lesson 391 of 3,5389. Dimensionality ReductionFree lesson

Standardization Before PCA

Why feature scaling matters: preventing features with large variance from dominating the principal components.

Standardization Before PCA

What You'll Learn

You'll understand why standardizing features before PCA prevents variables with large scales from unfairly dominating your principal components.

Why Feature Scaling Matters for PCA

Remember that PCA finds directions of maximum variance in your data. Here's the problem: variance is measured in the original units of each feature.

Imagine you have a dataset with two features:

  • House price (ranging from 100,000to100,000 to 1,000,000)
  • Number of bedrooms (ranging from 1 to 5)

The variance of house prices will be astronomically larger than the variance of bedrooms—not because price is more important, but simply because the numbers are bigger! When PCA searches for the direction of maximum variance, it will focus almost entirely on the house price feature and ignore bedrooms.

The Solution: Standardization

By standardizing your features (subtracting the mean and dividing by standard deviation), you transform all features to have:

  • Mean = 0
  • Standard deviation = 1

Now both features live on the same scale. PCA can fairly assess which directions capture the most meaningful variation in the data, rather than being misled by arbitrary units (dollars vs. counts).

The Analogy

Think of it like comparing noise levels. One person shouts their opinion in decibels, another whispers theirs in millibascal units. Even if both opinions are equally important, the shouter dominates the conversation—not because they're right, but because they're louder. Standardization gives everyone the same volume.

Key Takeaway: Always standardize your features before PCA unless you have a specific reason to preserve the original scale, because features with large variance will otherwise dominate the principal components regardless of their actual importance.