Course contentsShow
← Machine Learning and Deep Learning
Lesson 3306 of 3,53872. Bias and FairnessFree lesson

Reweighting Training Examples

Balance group representation by assigning different weights to training samples based on protected attributes.

Reweighting Training Examples

What you'll learn: How to balance group representation by giving under-represented or disadvantaged groups more influence during model training through example weighting.

The Core Idea

Imagine training a hiring model where 90% of historical hires were from Group A and only 10% from Group B. Your model will naturally learn patterns that favor Group A simply because those examples dominate the training data. Reweighting corrects this by assigning higher weights to underrepresented examples, forcing the model to pay more attention to them during optimization.

When computing loss during training, instead of treating all examples equally, you multiply each example's loss by a weight based on its protected attribute membership. Examples from underrepresented groups get weights > 1, while overrepresented groups get weights < 1.

How It Works

The standard loss becomes a weighted loss:

weighted_loss = Σ(weight_i × loss_i)

Common weighting schemes:

  • Inverse frequency: weight = 1 / (proportion of group in dataset)
  • Inverse square root: weight = 1 / sqrt(count of group)
  • Custom: Manually tune weights to achieve desired fairness metrics

This is a pre-processing technique that modifies the training objective without changing the model architecture or removing features.

Why It Helps

By reweighting, you're essentially telling the optimizer: "These minority group examples are just as important as the majority group." This can improve metrics like Demographic Parity or Equal Opportunity by reducing the model's bias toward overrepresented groups.

Key Takeaway: Reweighting training examples assigns higher importance to underrepresented groups during optimization, balancing their influence without discarding data or changing model architecture.