Standardization (Z-score Normalization)
What you'll learn: How to transform features so they have a mean of 0 and a standard deviation of 1, making them directly comparable.
The Core Idea
Standardization (also called Z-score normalization) transforms your features by centering them around zero and scaling them to have unit variance. Unlike Min-Max Normalization which squeezes data into a fixed range, standardization adjusts features based on how spread out they are from their average.
The formula is simple:
z = (x - mean) / standard_deviation
For each value in your feature, subtract the feature's mean, then divide by its standard deviation.
Why This Works
Think of it like grading on a curve. If one exam has scores averaging 70/100 and another averages 850/1000, raw scores aren't comparable. Standardization converts both to "how many standard deviations am I from the average?" — a universal scale.
After standardization:
- Mean becomes exactly 0
- Standard deviation becomes exactly 1
- Values typically range from roughly -3 to +3 (though not strictly bounded)
When to Use Standardization
Prefer standardization over Min-Max when:
- Your features contain outliers (standardization is less sensitive to them)
- You're using algorithms that assume normally distributed data (like linear regression, logistic regression)
- You need unbounded ranges (unlike Min-Max's 0-1 constraint)
Standardization is especially critical for algorithms you've learned that use distance or assume feature importance equality: KNN, K-Means, PCA, and SVM.
Key Takeaway: Standardization transforms features to mean 0 and standard deviation 1 using (x - mean) / std, making features comparable on a common scale while being robust to outliers—essential for distance-based and gradient-based algorithms.