Course contentsShow
← AI Engineering
Lesson 1568 of 1,88638. Bias, Fairness, and AlignmentFree lesson

Predictive Parity and Calibration

Measuring whether positive predictions have equal precision across groups.

Predictive Parity and Calibration

What You'll Learn

You'll understand how to measure whether positive predictions are equally reliable across different demographic groups.

The Core Idea

Predictive parity (also called outcome parity) requires that when your model predicts a positive outcome, it should be correct with equal probability across all groups. In other words, a positive prediction should mean the same thing regardless of who receives it.

Calibration is closely related: it means that when your model says "70% confident," it should actually be right 70% of the time — and this should hold consistently across groups.

Plain English Example

Imagine a lending model that approves loans. If it approves 100 Group A applicants and 80 actually repay (80% precision), and approves 100 Group B applicants where only 50 repay (50% precision), you have a predictive parity violation. The approval means different things for different groups — Group B is getting less reliable approvals.

Similarly, if your model outputs 90% confidence scores for both groups, but Group A's 90% predictions are correct 85% of the time while Group B's are correct only 60% of the time, your model is poorly calibrated across groups.

Why This Matters

Predictive parity ensures that your model's positive predictions are equally trustworthy regardless of demographic group. This is crucial when predictions carry consequences — like loan approvals, hiring decisions, or risk assessments. Poor calibration can lead to overconfidence in predictions for certain groups while underestimating risk in others.

Key Takeaway: Predictive parity measures whether positive predictions have equal precision (true positives / all positives) across groups, while calibration ensures confidence scores reflect actual accuracy consistently for all demographics.