What You'll Learn
You'll learn how the Yeo-Johnson transformation extends the Box-Cox method to handle datasets containing zero and negative values, making it more flexible for real-world data.
The Challenge with Box-Cox
In the previous lesson, you learned about the Box-Cox transformation for normalizing skewed data. However, Box-Cox has a critical limitation: it only works with strictly positive values. In real-world datasets, you often encounter zeros (like "no sales on a day") or negative values (like financial losses or temperature below zero). That's where Yeo-Johnson comes to the rescue.
How Yeo-Johnson Works
The Yeo-Johnson transformation uses different formulas depending on whether your data values are positive, zero, or negative. Think of it as having a Swiss Army knife instead of a single-purpose tool:
- For positive values: It applies a transformation similar to Box-Cox
- For zero and negative values: It uses a different mathematical formula that handles these cases safely
Like Box-Cox, Yeo-Johnson has a parameter (lambda, λ) that determines the strength and type of transformation. The algorithm tests different λ values to find the one that makes your data most closely resemble a normal distribution.
When to Use Yeo-Johnson
Choose Yeo-Johnson over Box-Cox when:
- Your dataset contains zeros
- Your dataset contains negative values
- You're unsure whether all values are positive
It's essentially a "safer" choice that works in all situations where Box-Cox works, plus many more.
Key Takeaway: Yeo-Johnson transformation is a more flexible alternative to Box-Cox that handles positive, zero, and negative values, making it ideal for real-world datasets where negative values or zeros are common.