From Regression to Classification
What You'll Learn
You'll discover why the linear regression tools we've been using can't handle classification tasks properly, and why we need a fundamentally different approach.
The Problem with Linear Regression for Classification
Imagine you're trying to predict whether an email is spam (1) or not spam (0) based on certain features. Your instinct might be to use linear regression since you've just mastered it—but there's a critical problem.
Linear regression produces unbounded outputs. If you fit a line to your data, predictions can be any value: -5.2, 0.7, 3.8, 100... anything! But classification needs discrete categories (0 or 1 for binary classification).
Why This Breaks Down
Even if you round predictions to 0 or 1, linear regression makes dangerous assumptions:
- It treats the "distance" between classes as meaningful (but "twice as spam" makes no sense)
- Outliers can drastically skew the decision boundary
- Predictions outside [0, 1] are nonsensical for categories
Think of it like using a thermometer to measure whether a light is on or off—the tool is designed for continuous measurements, not binary states.
What We Actually Need
For classification, we need:
- Bounded outputs between 0 and 1
- Probability interpretation: "70% likely to be spam"
- Non-linear decision boundaries that naturally separate classes
Instead of predicting the class directly, we'll predict the probability of belonging to a class. This shift from "what value?" to "how likely?" is the bridge from regression to classification.
Key Takeaway: Linear regression fails for classification because it produces unbounded outputs and treats classes as continuous values; classification requires probability-based approaches that output values between 0 and 1.