Course contentsShow
← Data Science
Lesson 1890 of 2,14544. Data Ethics and PrivacyFree lesson

Measuring Disparate Impact

Use statistical tests and metrics to detect when outcomes disproportionately affect certain groups, even without explicit bias.

Measuring Disparate Impact

What you'll learn: How to quantify whether your model or process disproportionately harms certain groups, even when you haven't explicitly used protected attributes.

The Core Idea

Disparate impact occurs when a seemingly neutral decision-making process produces significantly different outcomes for different groups. You might not intentionally discriminate, but if your hiring algorithm accepts 80% of male applicants and only 40% of female applicants, you've created disparate impact.

The challenge is measuring this quantitatively so you can act on it.

The Four-Fifths Rule (80% Rule)

The most common standard comes from U.S. employment law: if the selection rate for a protected group is less than 80% of the rate for the majority group, disparate impact may exist.

Formula:

Disparate Impact Ratio = (Selection Rate for Protected Group) / (Selection Rate for Reference Group)

If this ratio < 0.8, you have a red flag.

Example: Your loan approval model accepts 60% of white applicants and 45% of Black applicants.

  • Ratio = 0.45 / 0.60 = 0.75 (below 0.8 threshold → potential disparate impact)

Beyond the Four-Fifths Rule

Statistical tests help confirm whether differences are significant:

  • Chi-square tests compare outcome distributions across groups
  • Z-tests for proportions determine if selection rate differences are statistically meaningful
  • Adverse Impact Ratio extends the 80% rule with confidence intervals

Think of it like quality control: you're checking if the "defect rate" (adverse outcomes) differs systematically by group membership.

Why This Matters

Even without using protected attributes directly, proxy variables (ZIP code, credit history) can perpetuate historical inequities. Measuring disparate impact reveals hidden problems before they cause real-world harm.

Key Takeaway: Disparate impact metrics like the four-fifths rule and statistical significance tests let you detect when neutral-seeming processes systematically disadvantage protected groups, enabling you to fix fairness issues proactively.