Demographic and Protected Attribute Slices
What you'll learn: How to evaluate model performance across demographic groups to uncover hidden fairness issues and representation gaps that aggregate metrics conceal.
The Core Idea
When you evaluate a model's overall accuracy or F1 score, you're looking at average performance across your entire test set. But what if your model works brilliantly for one demographic group while failing catastrophically for another? Demographic and protected attribute slices are subdivisions of your data based on sensitive characteristics like race, gender, age, disability status, or socioeconomic indicators.
Think of it like a restaurant's average rating: a 4-star average might hide that vegetarian diners consistently have terrible experiences while meat-eaters love it. You need to "slice" the reviews by dietary preference to see the full picture.
Why This Matters
Protected attributes are legally and ethically significant. A hiring algorithm that performs well overall but systematically underperforms for certain racial groups isn't just unfair—it may be illegal and harmful. By creating slices based on these attributes, you can:
- Detect disparate impact: Identify when a model's error rates differ significantly across groups
- Find representation gaps: Discover if certain demographics are underrepresented in your data
- Measure fairness metrics: Calculate group-specific precision, recall, or false positive rates
- Meet compliance requirements: Satisfy regulatory standards for algorithmic fairness
In Practice
You define slices using the demographic attributes already discussed in "Defining Data Slices." For each slice (e.g., age_group=senior, gender=female), compute your task-specific metrics separately. Compare performance across slices to identify disparities that demand investigation and remediation.
Key Takeaway: Demographic slices transform fairness from an abstract principle into measurable reality—revealing when your model serves some groups far better than others, even when aggregate metrics look acceptable.