Course contentsShow
← Machine Learning and Deep Learning
Lesson 3303 of 3,53872. Bias and FairnessFree lesson

Computing Fairness Metrics with Fairlearn and AIF360

Practical implementation of bias measurement using standard Python fairness libraries.

Computing Fairness Metrics with Fairlearn and AIF360

You'll learn: How to use Python libraries to compute the fairness metrics you've studied in real ML projects.

Why Libraries Matter

Now that you understand metrics like demographic parity, equalized odds, and predictive parity, you need practical tools to compute them. Two leading Python libraries—Fairlearn and AIF360 (AI Fairness 360)—provide standardized implementations so you don't need to code metrics from scratch every time.

Fairlearn

Fairlearn (from Microsoft) focuses on scikit-learn compatibility and simplicity. It provides:

  • MetricFrame: Groups standard metrics (accuracy, precision) by sensitive features
  • Pre-built fairness metrics like demographic_parity_difference() and equalized_odds_difference()
  • Mitigation algorithms (covered later)

Example workflow:

from fairlearn.metrics import MetricFrame, selection_rate
import pandas as pd

# Group metrics by sensitive attribute
mf = MetricFrame(
    metrics=selection_rate,
    y_true=y_test,
    y_pred=predictions,
    sensitive_features=sensitive_attr
)

print(mf.by_group)  # Selection rate per group
print(mf.difference())  # Demographic parity difference

AIF360

IBM's AIF360 offers broader coverage of fairness concepts, including individual fairness measures and more exotic metrics. It uses its own dataset format and provides:

  • Metrics for both binary and continuous predictions
  • Counterfactual fairness checks
  • Advanced bias detection tools

Both libraries compute the same core metrics but with different APIs and ecosystems.

Key Takeaway: Fairlearn and AIF360 turn theoretical fairness metrics into actionable measurements—use Fairlearn for quick scikit-learn integration, AIF360 for comprehensive bias audits across multiple definitions of fairness.