From Linear to Logistic: Binary Classification
What You'll Learn
You'll discover why linear regression breaks down for yes/no problems and how logistic regression fixes this with a probability-based approach.
Why Linear Regression Fails for Classification
Linear regression predicts continuous values on an unlimited range—think house prices from $0 to infinity. But binary classification needs a yes or no answer: Is this email spam? Will the customer buy?
If you force linear regression into this task, you get nonsensical predictions. Imagine predicting "spam probability":
- Linear regression might output
-0.3(impossible—probabilities can't be negative!) - Or
1.8(also impossible—probabilities max out at 1.0) - Worse, a single outlier can dramatically shift your decision boundary
The core problem: Linear regression's output isn't constrained to [0, 1], and it treats the problem as if the distance between 0 and 1 is meaningful continuous space—but "not spam" and "spam" are discrete categories, not points on a ruler.
How Logistic Regression Solves This
Logistic regression takes your linear model's output and squashes it through a special function (the sigmoid/logistic function) that always produces values between 0 and 1. Now your output is a valid probability!
Instead of predicting the class directly, you predict: "What's the probability this belongs to class 1?"
- Output > 0.5? Predict class 1
- Output ≤ 0.5? Predict class 0
This transformation respects that you're dealing with categories, not a number line, while still using the power of linear combinations of features under the hood.
Key Takeaway: Linear regression fails at binary classification because it outputs unbounded continuous values, while logistic regression constrains outputs to [0, 1] by applying a sigmoid transformation, producing interpretable probabilities for discrete class membership.