Course contentsShow
← Machine Learning and Deep Learning
Lesson 263 of 3,5384. Classification with Logistic RegressionFree lesson

Multinomial Logistic Regression Model

Extending logistic regression to K classes with K weight vectors, predicting class probabilities via softmax activation.

Multinomial Logistic Regression Model

What You'll Learn

You'll learn how to extend logistic regression from two classes to K classes by using K weight vectors and the softmax activation function to produce class probabilities.

From Binary to Multiple Classes

In binary logistic regression, you learned one weight vector to separate two classes. Multinomial logistic regression scales this idea: instead of one set of weights, you maintain K separate weight vectors—one for each of the K classes you want to predict.

How It Works

For each class k, you compute a linear score (also called a "logit"):

z_k = w_k · x + b_k

This gives you K scores, one per class. But raw scores aren't probabilities! That's where softmax comes in. You already learned that softmax converts a vector of scores into a probability distribution. The model applies softmax to all K scores simultaneously:

P(class = k | x) = softmax(z)_k

Each weight vector essentially "votes" for its class. The softmax ensures the votes become proper probabilities that sum to 1.

The Complete Model

Think of it like K parallel binary classifiers, but they're all trained jointly so their outputs are calibrated relative to each other. During training, you adjust all K weight vectors simultaneously using gradient descent and a loss function (cross-entropy) that compares predicted probabilities to the true class.

Key Takeaway: Multinomial logistic regression extends binary classification to K classes by learning K weight vectors and using softmax activation to convert their linear outputs into a probability distribution over all classes.