Trade-offs Between Fairness and Accuracy
What you'll learn: How fairness interventions typically reduce model accuracy and how to navigate the accuracy-fairness tradeoff space.
When you apply fairness interventions—whether through reweighting, adversarial debiasing, fairness constraints, or post-processing adjustments—you almost always sacrifice some predictive accuracy. This isn't a bug; it's a fundamental tension. The model that maximizes accuracy alone often exploits patterns correlated with protected attributes, while fairness requires suppressing or balancing those patterns.
The Pareto Frontier
Think of this like fuel efficiency versus speed in cars: improving one often costs you the other. In ML, we can plot accuracy on one axis and fairness (measured by demographic parity difference, equalized odds, etc.) on the other. The Pareto frontier represents the best possible combinations—points where you can't improve fairness without losing accuracy, or vice versa.
Different mitigation strategies land at different points on this curve. Reweighting might give you 92% accuracy with 5% demographic parity difference, while threshold optimization might yield 90% accuracy but only 2% disparity. Neither dominates the other; they're different tradeoffs.
Quantifying the Cost
To navigate this space, you must:
- Measure baseline accuracy and fairness metrics before intervention
- Apply various mitigation techniques with different hyperparameter settings
- Plot the resulting accuracy-fairness pairs
- Choose a point that aligns with stakeholder values and legal requirements
There's no universal "right" answer—the tradeoff depends on context. A hiring algorithm might accept larger accuracy drops than a medical diagnosis system, depending on ethical priorities and domain constraints.
Key Takeaway: Fairness interventions create an accuracy-fairness Pareto frontier; understanding and navigating this tradeoff space requires quantifying both dimensions and making value-based decisions about acceptable performance costs.