StandardScaler and Feature Scaling
What you'll learn: How to standardize your features so they play nicely together in ML algorithms.
Why Scaling Matters
Imagine you're comparing people using two measurements: height in centimeters (150-200) and weight in kilograms (50-100). Now add income in dollars (20,000-150,000). Without scaling, income completely dominates any distance-based calculation simply because its numbers are larger—not because it's more important.
Many ML algorithms calculate distances between data points (like k-nearest neighbors) or use gradient descent optimization. When features live on wildly different scales, these algorithms struggle. A tiny change in income creates a huge distance, while large changes in height barely register.
What StandardScaler Does
StandardScaler transforms each feature to have:
- Zero mean (μ = 0): The average value becomes 0
- Unit variance (σ² = 1): The standard deviation becomes 1
The transformation formula: (x - mean) / standard_deviation
This centers all features around zero and spreads them similarly, making them directly comparable.
When You Need It
Feature scaling is critical for:
- Distance-based algorithms (k-NN, k-means clustering)
- Gradient descent optimization (linear/logistic regression, neural networks)
- Algorithms that assume features have similar scales (PCA, SVM)
You typically don't need it for tree-based algorithms (decision trees, random forests), which split on individual feature thresholds.
The Workflow
Fit the scaler on your training data, then transform both training and test sets using those same learned parameters. Never fit on test data—that would leak information from your test set!
Key Takeaway: StandardScaler puts all features on the same playing field by standardizing to zero mean and unit variance, preventing features with larger numerical ranges from dominating distance calculations and gradient descent.