Course contentsShow
← Machine Learning and Deep Learning
Lesson 437 of 3,53810. Feature EngineeringFree lesson

Multivariate Outlier Detection

Applying isolation forests, local outlier factor, and Mahalanobis distance to detect outliers in high-dimensional space.

Multivariate Outlier Detection

What You'll Learn

You'll learn how to detect outliers when working with many features at once, using techniques that understand complex relationships between variables.

The Challenge of Multiple Dimensions

Previously, you detected outliers by examining one feature at a time—looking at heights, weights, or ages individually. But what if someone has a combination of features that's unusual? A basketball player might be tall (normal) and heavy (normal) separately, but their height-weight combination could still be typical for athletes. Conversely, someone with "normal" values individually might have an abnormal combination.

Multivariate outlier detection finds data points that are unusual when considering all features together.

Three Powerful Approaches

Isolation Forest

Think of this as isolating the "weird kid" in fewer steps. Normal points are clustered together and take many random splits to isolate. Outliers are alone and get isolated quickly with fewer splits. It works brilliantly in high-dimensional spaces because outliers naturally separate themselves.

Local Outlier Factor (LOF)

This method asks: "How dense is the neighborhood around each point?" If your neighbors are packed tightly but you're far away, you're an outlier. LOF compares each point's local density to its neighbors' densities—points in sparse regions relative to surroundings get flagged.

Mahalanobis Distance

Imagine measuring distance while accounting for how features vary and correlate. Unlike simple Euclidean distance, Mahalanobis distance stretches and rotates the space based on your data's covariance structure. A point might be close in raw distance but far in "statistical distance."

When to Use What

  • Isolation Forest: Fast, scalable, works with minimal assumptions
  • LOF: Detects local density anomalies, great for varying cluster densities
  • Mahalanobis Distance: Assumes roughly Gaussian data, sensitive to feature correlations

Key Takeaway: Multivariate outlier detection considers all features simultaneously to find unusual combinations that univariate methods miss, using algorithms optimized for high-dimensional space.