Threshold Optimization
You'll learn how to adjust decision thresholds independently for different demographic groups to achieve fairness goals like equal opportunity or equalized odds.
Core Idea
Most classifiers output probabilities, which you convert to binary decisions using a threshold (often 0.5). But using the same threshold for all groups can produce unfair outcomes. Threshold optimization means setting different thresholds for different protected groups to satisfy fairness criteria.
Think of it like adjusting the "passing grade" differently for groups that faced different levels of difficulty on a test—not to favor anyone unfairly, but to correct for systematic disadvantages in how the test was administered.
How It Works
Suppose your model predicts loan approval. After training, you might discover:
- Group A: threshold 0.5 yields 70% true positive rate (TPR)
- Group B: threshold 0.5 yields 50% TPR
For equal opportunity, you want equal TPR across groups. You could:
- Lower Group B's threshold to 0.35, raising their TPR to ~70%
- Keep Group A's threshold at 0.5
For equalized odds, you'd also equalize false positive rates (FPR) by adjusting thresholds to balance both metrics simultaneously.
Why This Matters
Threshold optimization is a post-processing technique—it doesn't retrain your model. You fit once, then find group-specific thresholds that meet your fairness definition. It's computationally cheap and interpretable, though it requires knowing group membership at decision time.
Key Takeaway: Threshold optimization mitigates bias by applying different decision boundaries to different groups, achieving fairness metrics like equal opportunity without retraining your model.