Log Loss (Cross-Entropy Loss)
You'll learn: How to evaluate probabilistic classifiers by measuring how confidently wrong (or right) their predictions are.
What Is Log Loss?
Log loss, also called cross-entropy loss, measures the quality of probability predictions. Unlike accuracy (which only cares if you got the class right), log loss cares about how confident your model was in its prediction.
If your model predicts 90% probability for class A when the true answer is class B, log loss penalizes this heavily—you were very wrong and very confident about it. But if you predicted 55% for A when B was correct, the penalty is lighter because you were less certain.
How It Works
For a single binary prediction:
Log Loss = -[y × log(p) + (1-y) × log(1-p)]
Where:
yis the true label (0 or 1)pis the predicted probability for class 1
Key properties:
- Lower log loss = better model
- Perfect predictions (p=1 when y=1, or p=0 when y=0) give log loss of 0
- Confident wrong predictions approach infinite penalty
- Multi-class version averages loss across all classes
Analogy
Imagine a weather forecaster predicting rain probability. If they say "95% chance of rain" and it's sunny, they're penalized heavily for overconfidence. If they say "55% rain" and it's sunny, the penalty is much lighter—they showed appropriate uncertainty.
Key Takeaway: Log loss evaluates probabilistic predictions by heavily penalizing confident mistakes, making it ideal for models that output class probabilities rather than hard labels.