Course contentsShow
← Machine Learning and Deep Learning
Lesson 3320 of 3,53872. Bias and FairnessFree lesson

Disaggregated Performance Analysis

Computing model performance metrics separately for each demographic group to reveal disparate impact patterns.

Disaggregated Performance Analysis

What you'll learn: How to compute performance metrics separately for each demographic group to uncover hidden fairness problems in your model.

The Core Idea

Disaggregated performance analysis means breaking down your model's overall performance metrics by protected attributes—computing accuracy, precision, recall, false positive rate, and other metrics separately for each demographic subgroup rather than just looking at aggregate numbers.

Think of it like a restaurant chain checking its average customer satisfaction score. A 4.2/5 overall rating looks great, but if you disaggregate by location and find one branch consistently scores 2.5 while others score 4.8, you've discovered a serious problem that the aggregate hid.

Why This Matters in Fairness Audits

A model might show 90% accuracy overall, but when you disaggregate:

  • Group A: 95% accuracy
  • Group B: 78% accuracy

This disparity reveals that the model performs significantly worse for Group B—a form of disparate impact that aggregate metrics completely obscure.

During a fairness audit, you systematically compute metrics like:

  • True/false positive rates per group
  • Precision and recall per group
  • Error distribution across demographics
  • Confusion matrix breakdowns by protected attribute

Real-World Application

In medical diagnosis, a model might seem highly accurate overall but fail disproportionately for underrepresented groups. Disaggregated analysis exposes where your model's blind spots are, which groups face higher misclassification rates, and whether error types (false positives vs. false negatives) differ across demographics—all critical for addressing bias.

Key Takeaway: Disaggregated performance analysis reveals disparate impact by computing metrics separately for each demographic group, exposing fairness issues that aggregate performance numbers hide.