Fairness-Accuracy Tradeoffs
You'll learn: Why making models fairer often requires accepting lower accuracy, and how to navigate this ethical tension.
The Core Tension
When you measure disparate impact and find unfairness, fixing it rarely comes for free. There's a mathematical tradeoff: optimizing purely for accuracy often creates unfair outcomes, while enforcing fairness constraints typically reduces overall accuracy.
Here's why: Machine learning models maximize predictive performance on historical data. If that data reflects historical biases (which we know it does), the most "accurate" model will learn and perpetuate those patterns. A model predicting loan default might achieve 92% accuracy by relying heavily on zip code—a proxy variable for race. Removing or downweighting that feature might drop accuracy to 88%, but produce fairer outcomes across protected classes.
Navigating the Tradeoff
Think of it like adjusting a stereo's bass and treble—boosting one often means sacrificing the other. You must decide:
-
What level of accuracy loss is acceptable? A 1% drop in medical diagnosis accuracy has different stakes than 1% in ad targeting.
-
Which fairness metric matters most? Equalizing false positive rates across groups may conflict with equalizing false negative rates—you often can't optimize both simultaneously.
-
Who bears the cost? Sometimes the aggregate accuracy loss is small, but one subgroup's performance drops significantly.
The Practical Reality
This isn't about choosing "fairness OR accuracy"—it's about finding an acceptable balance. Document your constraints, measure disparate impact across the metrics that matter for your use case, and make transparent decisions about where you land on the tradeoff curve.
Key Takeaway: Fairness and accuracy often pull in opposite directions mathematically; data scientists must make informed, documented decisions about where to balance these competing objectives based on the ethical stakes and real-world consequences.