Course contentsShow
← Machine Learning and Deep Learning
Lesson 466 of 3,53811. Model Evaluation MetricsFree lesson

Log Loss (Cross-Entropy Loss)

Evaluating probabilistic predictions by penalizing confident wrong predictions heavily.

Log Loss (Cross-Entropy Loss)

You'll learn: How to evaluate probabilistic classifiers by measuring how confidently wrong (or right) their predictions are.

What Is Log Loss?

Log loss, also called cross-entropy loss, measures the quality of probability predictions. Unlike accuracy (which only cares if you got the class right), log loss cares about how confident your model was in its prediction.

If your model predicts 90% probability for class A when the true answer is class B, log loss penalizes this heavily—you were very wrong and very confident about it. But if you predicted 55% for A when B was correct, the penalty is lighter because you were less certain.

How It Works

For a single binary prediction:

Log Loss = -[y × log(p) + (1-y) × log(1-p)]

Where:

  • y is the true label (0 or 1)
  • p is the predicted probability for class 1

Key properties:

  • Lower log loss = better model
  • Perfect predictions (p=1 when y=1, or p=0 when y=0) give log loss of 0
  • Confident wrong predictions approach infinite penalty
  • Multi-class version averages loss across all classes

Analogy

Imagine a weather forecaster predicting rain probability. If they say "95% chance of rain" and it's sunny, they're penalized heavily for overconfidence. If they say "55% rain" and it's sunny, the penalty is much lighter—they showed appropriate uncertainty.

Key Takeaway: Log loss evaluates probabilistic predictions by heavily penalizing confident mistakes, making it ideal for models that output class probabilities rather than hard labels.