Distribution Checks for All Variables
What you'll learn: How to systematically examine the distribution of every variable in your dataset using visual and statistical tools to understand data behavior before modeling.
Why Check Distributions?
Think of distributions as the "personality" of each variable. Just like you'd want to know someone's temperament before working with them, you need to understand how your data behaves—whether it's normally distributed, skewed, uniform, or bimodal. This shapes every decision downstream, from choosing transformations to selecting appropriate statistical tests.
The Three-Pronged Approach
1. Visual Inspection
Create histograms for numeric variables to see shape, symmetry, and potential outliers at a glance. For categorical variables, use bar charts to see frequency distributions. These give you the "gestalt"—the immediate feel of the data.
2. Summary Statistics
Compute mean, median, standard deviation, min, max, quartiles for numeric data. For categorical data, examine counts and percentages. When mean and median differ significantly, you've spotted skewness. When the range is surprisingly wide, you've found potential outliers.
3. Statistical Tests
Apply formal tests like Shapiro-Wilk or Kolmogorov-Smirnov to check if numeric distributions match theoretical ones (like normal distribution). These provide objective confirmation of what your eyes suggest.
What You're Looking For
- Shape: Normal, skewed left/right, bimodal?
- Spread: Narrow or wide variance?
- Outliers: Extreme values that stand out?
- Missing patterns: Gaps or unusual concentrations?
Check every variable systematically—don't skip any. Surprises in distributions often reveal data quality issues or important domain insights.
Key Takeaway: Distribution checks combine histograms, summary statistics, and formal tests to reveal each variable's behavior, informing transformation needs and analytical approach before any modeling begins.