From Linear Regression to Classification
You'll learn why we can't just use linear regression for classification tasks and what we need instead.
The Problem with Linear Regression for Classification
You've mastered linear regression, which predicts continuous values like house prices or temperatures. But what happens when you try to predict whether an email is spam (1) or not spam (0)?
Linear regression gives you a straight line that can output any number: -5, 0.3, 2.8, 147... anything! When you're trying to predict a class label (0 or 1), these unbounded predictions create serious problems:
- A prediction of 3.5 for "spam" doesn't make sense
- A prediction of -0.8 for "not spam" is impossible to interpret
- You'd need to pick arbitrary thresholds to convert these strange numbers into classes
Why We Need Bounded Predictions
Think of classification as answering "What's the probability this email is spam?" Probabilities must be between 0 and 1 by definition. You can't have a 150% chance of something happening, and you can't have a -30% chance either.
A linear regression line doesn't respect these boundaries. As your input features grow larger or smaller, the line shoots off to infinity in both directions. This makes it fundamentally unsuitable for expressing probabilities or class membership.
For classification to work properly, we need a function that:
- Takes any input and squashes the output between 0 and 1
- Can be interpreted as a probability
- Still maintains a smooth, differentiable shape we can optimize
Key Takeaway: Linear regression fails at classification because it produces unbounded predictions, but classification requires outputs between 0 and 1 to represent valid probabilities or class membership scores.