Probabilistic Interpretation and Odds
What you'll learn: How logistic regression outputs represent probabilities through log-odds and how to convert between probability and odds.
Understanding the Output
When logistic regression makes a prediction, it doesn't just say "yes" or "no"—it tells you how confident it is. The sigmoid function transforms the linear combination of features into a probability between 0 and 1. But what's actually happening inside?
From Probability to Odds
Odds express the ratio of success to failure. If something has a 75% chance of happening (probability = 0.75), the odds are 3:1, or simply 3.
odds = probability / (1 - probability)
For our 75% example: 0.75 / 0.25 = 3. Think of it like betting: "3 times more likely to happen than not."
The Log-Odds (Logit)
The log-odds (or logit) is just the natural logarithm of the odds:
log-odds = ln(odds) = ln(p / (1 - p))
Here's the key insight: logistic regression's linear part (before the sigmoid) outputs log-odds. When you compute w₀ + w₁x₁ + w₂x₂ + ..., that value is the log-odds!
The sigmoid function then converts this back to probability:
probability = 1 / (1 + e^(-log-odds))
Why This Matters
Log-odds can range from negative infinity to positive infinity, making them perfect for linear modeling. Probabilities are squeezed between 0 and 1. By working in log-odds space, logistic regression can use familiar linear combinations while ultimately producing valid probabilities.
Key Takeaway: Logistic regression outputs log-odds linearly, which the sigmoid converts to probabilities. Understanding this connection lets you interpret model coefficients as effects on odds rather than just abstract numbers.