Course contentsShow
← Computer Science
Lesson 2826 of 2,87263. Classical ML AlgorithmsFree lesson

From Linear to Logistic: Binary Classification

Understand why linear regression fails for classification and how logistic regression addresses binary outcomes.

From Linear to Logistic: Binary Classification

What You'll Learn

You'll discover why linear regression breaks down for yes/no problems and how logistic regression fixes this with a probability-based approach.

Why Linear Regression Fails for Classification

Linear regression predicts continuous values on an unlimited range—think house prices from $0 to infinity. But binary classification needs a yes or no answer: Is this email spam? Will the customer buy?

If you force linear regression into this task, you get nonsensical predictions. Imagine predicting "spam probability":

  • Linear regression might output -0.3 (impossible—probabilities can't be negative!)
  • Or 1.8 (also impossible—probabilities max out at 1.0)
  • Worse, a single outlier can dramatically shift your decision boundary

The core problem: Linear regression's output isn't constrained to [0, 1], and it treats the problem as if the distance between 0 and 1 is meaningful continuous space—but "not spam" and "spam" are discrete categories, not points on a ruler.

How Logistic Regression Solves This

Logistic regression takes your linear model's output and squashes it through a special function (the sigmoid/logistic function) that always produces values between 0 and 1. Now your output is a valid probability!

Instead of predicting the class directly, you predict: "What's the probability this belongs to class 1?"

  • Output > 0.5? Predict class 1
  • Output ≤ 0.5? Predict class 0

This transformation respects that you're dealing with categories, not a number line, while still using the power of linear combinations of features under the hood.

Key Takeaway: Linear regression fails at binary classification because it outputs unbounded continuous values, while logistic regression constrains outputs to [0, 1] by applying a sigmoid transformation, producing interpretable probabilities for discrete class membership.