The Sigmoid Function
What you'll learn: How the sigmoid function transforms any real number into a probability value between 0 and 1.
The Bridge from Linear to Probabilistic
Remember from "From Linear Regression to Classification" that we can't use raw linear regression outputs for classification—they range from negative infinity to positive infinity. But probabilities must live between 0 and 1. The sigmoid function (also called the logistic function) is the mathematical tool that solves this problem.
What the Sigmoid Does
The sigmoid function takes any real number as input and squashes it into the range (0, 1):
σ(z) = 1 / (1 + e^(-z))
Where:
zis any real number (like the output of a linear model:wx + b)eis Euler's number (~2.718)- The output is always between 0 and 1
How It Behaves
Think of the sigmoid like a smooth step function:
- Large positive numbers (z > 0): Output approaches 1
- Zero (z = 0): Output is exactly 0.5
- Large negative numbers (z < 0): Output approaches 0
The transition is smooth and S-shaped, which is why it's sometimes called the "S-curve."
Why This Matters for Classification
When we apply sigmoid to a linear model's output, we get a valid probability:
P(class = 1) = σ(wx + b)
This probability tells us how confident the model is that an input belongs to the positive class. If it outputs 0.85, there's an 85% predicted probability of class 1.
Key Takeaway: The sigmoid function is the critical link that transforms unbounded linear outputs into valid probabilities for binary classification, mapping any real number to a value between 0 and 1.