The Sigmoid Function
What you'll learn: How the sigmoid function transforms any real number into a probability-like value between 0 and 1 for binary classification.
Why We Need It
You've just learned that logistic regression solves binary classification problems. But there's a fundamental mismatch: the linear model equation you used in linear regression produces outputs from negative infinity to positive infinity, while classification requires probabilities between 0 and 1.
The sigmoid function solves this beautifully.
The Sigmoid (Logistic) Function
The sigmoid function (also called the logistic function) is defined as:
σ(z) = 1 / (1 + e^(-z))
Where z is any real number (often the output of your linear model: z = w₁x₁ + w₂x₂ + ... + b).
How It Works
Think of the sigmoid as a smooth "squashing" function:
- When
zis very negative (like -10),e^(-z)becomes huge, so σ(z) approaches 0 - When
zis very positive (like +10),e^(-z)approaches 0, so σ(z) approaches 1 - When
z = 0, σ(0) = 0.5 exactly
The function creates an S-shaped curve that gracefully transitions from 0 to 1, with the steepest change happening around z = 0.
Why It Matters for Classification
The sigmoid transforms your linear model's raw output into something interpretable as a probability: the probability that an example belongs to the positive class (class 1). If σ(z) = 0.85, you can interpret this as "85% confident this is class 1."
Key Takeaway: The sigmoid function maps any linear output to a probability between 0 and 1, making it the essential bridge from linear regression to binary classification in logistic regression.