In-processing: Fairness-Aware Training
What you'll learn: How to build fairness directly into your model's training process using constraints and specialized objectives.
The Core Idea
While pre-processing fixes your data before training, in-processing techniques modify the training process itself to encourage fair predictions. Think of it like training an athlete: instead of just selecting a balanced team (pre-processing), you're actively coaching them during practice to avoid biased behaviors.
The key approach is adding fairness constraints or penalties directly to your loss function. Your model must now optimize for both accuracy and fairness simultaneously.
How It Works
Adversarial Debiasing
Train two models at once: your main predictor and an "adversary" that tries to guess protected attributes (like gender or race) from your predictor's outputs. The predictor learns to make accurate predictions that don't leak information the adversary could use—forcing fair representations.
It's like playing a game where you must win at your task while hiding clues about sensitive attributes.
Fairness Regularization
Add a fairness penalty term to your loss function. For example:
Total Loss = Prediction Loss + λ × Fairness Penalty
The λ parameter controls the fairness-accuracy trade-off. Higher values enforce stricter fairness but may reduce overall accuracy (remember fairness-accuracy trade-offs from lesson 1571).
The fairness penalty might measure demographic parity violations, equalized odds gaps, or other metrics you've already learned.
Why This Matters
In-processing is powerful because it addresses bias during learning, not before or after. The model learns representations that are inherently more fair, rather than just correcting biased outputs later.
Key Takeaway: In-processing embeds fairness directly into model optimization through adversarial training or regularization penalties, creating models that balance accuracy with fairness constraints during the learning process itself.