Course contentsShow
← Machine Learning and Deep Learning
Lesson 409 of 3,53810. Feature EngineeringFree lesson

Standardization (Z-score Normalization)

Centering features to mean 0 and scaling to unit variance using mean and standard deviation.

Standardization (Z-score Normalization)

What you'll learn: How to transform features so they have a mean of 0 and a standard deviation of 1, making them directly comparable.

The Core Idea

Standardization (also called Z-score normalization) transforms your features by centering them around zero and scaling them to have unit variance. Unlike Min-Max Normalization which squeezes data into a fixed range, standardization adjusts features based on how spread out they are from their average.

The formula is simple:

z = (x - mean) / standard_deviation

For each value in your feature, subtract the feature's mean, then divide by its standard deviation.

Why This Works

Think of it like grading on a curve. If one exam has scores averaging 70/100 and another averages 850/1000, raw scores aren't comparable. Standardization converts both to "how many standard deviations am I from the average?" — a universal scale.

After standardization:

  • Mean becomes exactly 0
  • Standard deviation becomes exactly 1
  • Values typically range from roughly -3 to +3 (though not strictly bounded)

When to Use Standardization

Prefer standardization over Min-Max when:

  • Your features contain outliers (standardization is less sensitive to them)
  • You're using algorithms that assume normally distributed data (like linear regression, logistic regression)
  • You need unbounded ranges (unlike Min-Max's 0-1 constraint)

Standardization is especially critical for algorithms you've learned that use distance or assume feature importance equality: KNN, K-Means, PCA, and SVM.

Key Takeaway: Standardization transforms features to mean 0 and standard deviation 1 using (x - mean) / std, making features comparable on a common scale while being robust to outliers—essential for distance-based and gradient-based algorithms.