In this lesson, you will learn how to evaluate the "shape" of your numeric data distributions using Skewness and Kurtosis to ensure your statistical insights are based on a healthy dataset.
When you aggregate data, like calculating an average or a trend, you often assume your data behaves predictably. However, if your data is "lopsided" or contains extreme outliers, those simple aggregates can become misleading. We use Skewness and Kurtosis to quantify these deviations.
Understanding Skewness
Skewness tells us about the asymmetry of your data distribution.
- Zero skew: The distribution is perfectly symmetrical.
- Positive skew: A long tail on the right side (e.g., household income, where most people earn less but a few earn millions).
- Negative skew: A long tail on the left side (e.g., test scores in a very easy exam, where most people score high).
Understanding Kurtosis
Kurtosis describes the "tailedness" or the presence of outliers. It tells you how much data is in the extreme tails versus the center.
- High Kurtosis: Indicates "heavy tails," meaning your dataset has frequent, extreme outliers that could skew your results.
- Low Kurtosis: Indicates a "flat" distribution with fewer extreme outliers.
SQL Implementation Approach
Because standard SQL doesn't have a universal SKEW() function, we calculate these using the moments of the distribution (the difference between a value and the AVG). For a set of values x, Skewness involves calculating the average of (x - mean)^3, and Kurtosis involves (x - mean)^4.
For a basic sanity check, you can compare the AVG vs. the MEDIAN (which you learned to calculate using frequency analysis earlier). If the AVG is significantly higher than the MEDIAN, you likely have a Positive Skew.
-- Conceptual sanity check: Comparing mean to median
SELECT
AVG(order_amount) as mean_value,
PERCENTILE_CONT(0.5) WITHIN GROUP (ORDER BY order_amount) as median_value
FROM sales_data;
If the gap between these two is massive, your data isn't "normal." This is your "Data Quality Gate" warning you that simple averages might hide the truth behind your outliers.
Key Takeaway: Skewness reveals the tilt of your data, while Kurtosis warns you of extreme outliers; checking these ensures your statistical aggregates aren't being distorted by a non-normal distribution.