Course contentsShow
← Machine Learning and Deep Learning
Lesson 261 of 3,5384. Classification with Logistic RegressionFree lesson

The Softmax Function Definition

Deriving the softmax transformation that converts multiple logits into a valid probability distribution over K classes.

The Softmax Function Definition

What you'll learn: How the softmax function transforms raw model outputs (logits) into a valid probability distribution across multiple classes.


The Need for Softmax

When you have K classes (say, 3 or more), you need a way to convert your model's raw outputs into probabilities that sum to exactly 1.0. You've seen how the sigmoid function squashes a single value between 0 and 1 for binary classification. For multiclass problems, we need something that handles multiple outputs simultaneously while ensuring they form a valid probability distribution.

How Softmax Works

Given K raw scores (called logits) from your model — one per class — the softmax function does two things:

  1. Exponentiation: Converts each logit z_i to e^(z_i), making all values positive.
  2. Normalization: Divides each exponentiated value by the sum of all exponentiated values.

The formula for class i:

softmax(z_i) = e^(z_i) / (e^(z_1) + e^(z_2) + ... + e^(z_K))

An Analogy

Think of logits as raw "votes" for each class. Softmax converts these votes into percentages: the exponential emphasizes stronger votes, and the division ensures all percentages add up to 100%.

Example: If your model outputs logits [2.0, 1.0, 0.1] for three classes, softmax converts these to probabilities like [0.659, 0.242, 0.099] — they're all positive and sum to 1.

Why It Matters

Softmax is the natural multiclass extension of sigmoid. It produces interpretable probabilities and pairs perfectly with cross-entropy loss (which you'll use to train multiclass models).

Key Takeaway: The softmax function transforms K unconstrained logits into K probabilities that are positive and sum to 1, enabling principled multiclass classification.