Course contentsShow
← Machine Learning and Deep Learning
Lesson 3176 of 3,53869. LLM EvaluationFree lesson

Bradley-Terry Model for Rankings

Using the Bradley-Terry statistical model to convert pairwise preferences into global model rankings.

Bradley-Terry Model for Rankings

What you'll learn: How the Bradley-Terry statistical model transforms pairwise win/loss comparisons into a global ranking of LLM capabilities.

The Core Idea

After running many pairwise comparisons where human judges or LLM-judges pick winners between model outputs, you face a question: how do you aggregate thousands of individual matchups into a single, coherent ranking?

The Bradley-Terry model provides an elegant answer. It assumes each model has a latent "strength" parameter, and the probability that model i beats model j follows a simple logistic form:

P(i beats j) = strength_i / (strength_i + strength_j)

Given your collection of pairwise outcomes (Model A beat Model B 73 times out of 100, etc.), you use maximum likelihood estimation to solve for the strength parameters that best explain your observed win rates. These strengths directly translate into a global ranking.

Why This Matters

Unlike Elo ratings (which update sequentially based on match history), Bradley-Terry computes strengths simultaneously from all observed comparisons. This makes it ideal when you have a complete or near-complete matrix of pairwise results—common in controlled LLM evaluation where you can test all model pairs systematically.

The model handles intransitive outcomes gracefully (where A beats B, B beats C, but C beats A) by finding the maximum-likelihood strengths that best reconcile contradictory signals.

Connection to Win Rates

Your pairwise win rates from earlier lessons become the input data. The Bradley-Terry model converts those percentages into interpretable strength scores that preserve relative ordering and satisfy statistical properties.

Key Takeaway: The Bradley-Terry model transforms raw pairwise comparison data into statistically-grounded global rankings by estimating latent strength parameters that maximize the likelihood of observed match outcomes.