Group Fairness Metrics Overview
You'll learn how to measure whether your model treats different demographic groups fairly using quantitative statistical metrics.
What Are Group Fairness Metrics?
Group fairness metrics are mathematical measures that compare how your model's predictions affect different demographic groups defined by protected attributes (like race, gender, or age). Rather than evaluating individual predictions, these metrics look at aggregate statistics across entire subgroups in your population.
Think of it like evaluating a hiring system: you wouldn't just check one person's application—you'd examine whether men and women, or different racial groups, receive job offers at similar rates overall.
The Main Families of Group Metrics
You've already encountered the key group fairness definitions:
- Statistical Parity (Demographic Parity): Do all groups receive positive predictions at the same rate?
- Equal Opportunity: Among truly qualified individuals, do all groups have equal chances of being correctly identified?
- Equalized Odds: Does the model achieve equal true positive and false positive rates across groups?
- Predictive Parity: When the model predicts "positive," is it equally accurate across groups?
Each metric captures a different intuition about what "fair" means. Some focus on outcomes (who gets selected?), others on error rates (who gets wrongly rejected?), and still others on trust (if accepted, what's your actual chance of success?).
Why Multiple Metrics Matter
Unfortunately, you cannot satisfy all fairness metrics simultaneously (except in trivial cases)—this is the impossibility theorem you've learned. Your choice depends on your application's values: Is it more important that equally qualified people have equal chances, or that selection rates are balanced?
Key Takeaway: Group fairness metrics quantify demographic disparities in model behavior by comparing statistical measures across protected groups—choose metrics that align with your application's ethical priorities and legal requirements.