Misleading Aggregations and Binning
You'll learn how seemingly innocent decisions about grouping data or choosing bin sizes can hide real patterns or manufacture fake ones.
The Problem with Invisible Choices
Aggregation (averaging, summing, grouping) and binning (creating intervals like "0-10, 10-20") are essential tools, but they're also powerful filters that can reshape your data's story. When you average customer spending across a region, you might hide that 90% spend 500. When you bin ages into "18-65" and "65+", you erase the difference between an 18-year-old and a 60-year-old.
How It Misleads
Aggregation problems:
- Over-smoothing: Mean income by state masks extreme inequality within states
- Wrong summary: Using mean when data has outliers (better: median)
- Hidden subgroups: Averaging diverse populations together (Simpson's Paradox territory)
Binning problems:
- Too wide: Bins like "0-100" collapse all variation
- Too narrow: Creates noisy, overfit patterns from random variation
- Strategic boundaries: Choosing cutoffs that produce desired patterns rather than natural ones
- Equal-width vs equal-frequency: Different bin strategies tell different stories
Real Example
Imagine binning customer ages into just two groups: "Under 50" and "50+". You might conclude older customers spend more. But with finer bins (20-30, 30-40, etc.), you'd discover spending actually peaks at 40-45 then declines. The coarse binning manufactured a false pattern.
Key Takeaway: Aggregation and binning choices are analytical decisions, not neutral mechanics—always show your grouping choices, test sensitivity to different bin widths, and ask whether your summary method matches your data's actual structure.