Multinomial Logistic Regression Model
What You'll Learn
You'll learn how to extend logistic regression from two classes to K classes by using K weight vectors and the softmax activation function to produce class probabilities.
From Binary to Multiple Classes
In binary logistic regression, you learned one weight vector to separate two classes. Multinomial logistic regression scales this idea: instead of one set of weights, you maintain K separate weight vectors—one for each of the K classes you want to predict.
How It Works
For each class k, you compute a linear score (also called a "logit"):
z_k = w_k · x + b_k
This gives you K scores, one per class. But raw scores aren't probabilities! That's where softmax comes in. You already learned that softmax converts a vector of scores into a probability distribution. The model applies softmax to all K scores simultaneously:
P(class = k | x) = softmax(z)_k
Each weight vector essentially "votes" for its class. The softmax ensures the votes become proper probabilities that sum to 1.
The Complete Model
Think of it like K parallel binary classifiers, but they're all trained jointly so their outputs are calibrated relative to each other. During training, you adjust all K weight vectors simultaneously using gradient descent and a loss function (cross-entropy) that compares predicted probabilities to the true class.
Key Takeaway: Multinomial logistic regression extends binary classification to K classes by learning K weight vectors and using softmax activation to convert their linear outputs into a probability distribution over all classes.