Course contentsShow
← Machine Learning and Deep Learning
Lesson 534 of 3,53812. Model Validation and TuningFree lesson

Isotonic Regression for Calibration

Non-parametric calibration: using isotonic regression to learn monotonic mapping from scores to probabilities.

Isotonic Regression for Calibration

What you'll learn: How to use isotonic regression as a flexible, non-parametric method to calibrate your model's predicted probabilities without assuming any specific functional form.

The Core Idea

While Platt Scaling fits a specific parametric function (logistic) to map your model's scores to calibrated probabilities, isotonic regression takes a more flexible approach. It learns a monotonic (non-decreasing) step function that maps uncalibrated scores to calibrated probabilities directly from data, making no assumptions about the relationship's shape.

Think of it like building a staircase: isotonic regression groups your predictions into bins and adjusts each bin's height to ensure that predicted probabilities match actual frequencies, with the constraint that going up the stairs (higher predicted scores) never leads downward (lower calibrated probabilities).

When to Use It

Isotonic regression shines when:

  • Your calibration curve isn't sigmoid-shaped (Platt Scaling's assumption)
  • You have sufficient calibration data (it's more data-hungry than Platt Scaling)
  • Your model produces rankings that are good but probabilities that are systematically wrong in complex ways

The Trade-off

Because isotonic regression is non-parametric, it's more flexible but also more prone to overfitting on small calibration sets. It essentially memorizes the calibration data's shape, which is powerful when you have enough examples but risky otherwise.

The monotonicity constraint prevents complete chaos—it ensures that if model A predicts higher than model B, the calibrated probability for A won't be lower than B's—but you still need hundreds or thousands of calibration examples for stable results.

Key Takeaway: Isotonic regression provides flexible, non-parametric calibration by learning a monotonic mapping from scores to probabilities, ideal when calibration curves are non-sigmoid but requires more data than parametric methods like Platt Scaling.