Course contentsShow
← Machine Learning and Deep Learning
Lesson 3130 of 3,53868. Evaluation in PracticeFree lesson

Demographic and Protected Attribute Slices

Evaluating performance across demographic groups to detect fairness issues and representation gaps.

Demographic and Protected Attribute Slices

What you'll learn: How to evaluate model performance across demographic groups to uncover hidden fairness issues and representation gaps that aggregate metrics conceal.

The Core Idea

When you evaluate a model's overall accuracy or F1 score, you're looking at average performance across your entire test set. But what if your model works brilliantly for one demographic group while failing catastrophically for another? Demographic and protected attribute slices are subdivisions of your data based on sensitive characteristics like race, gender, age, disability status, or socioeconomic indicators.

Think of it like a restaurant's average rating: a 4-star average might hide that vegetarian diners consistently have terrible experiences while meat-eaters love it. You need to "slice" the reviews by dietary preference to see the full picture.

Why This Matters

Protected attributes are legally and ethically significant. A hiring algorithm that performs well overall but systematically underperforms for certain racial groups isn't just unfair—it may be illegal and harmful. By creating slices based on these attributes, you can:

  • Detect disparate impact: Identify when a model's error rates differ significantly across groups
  • Find representation gaps: Discover if certain demographics are underrepresented in your data
  • Measure fairness metrics: Calculate group-specific precision, recall, or false positive rates
  • Meet compliance requirements: Satisfy regulatory standards for algorithmic fairness

In Practice

You define slices using the demographic attributes already discussed in "Defining Data Slices." For each slice (e.g., age_group=senior, gender=female), compute your task-specific metrics separately. Compare performance across slices to identify disparities that demand investigation and remediation.

Key Takeaway: Demographic slices transform fairness from an abstract principle into measurable reality—revealing when your model serves some groups far better than others, even when aggregate metrics look acceptable.