Decision Boundaries in Logistic Regression
What you'll learn: How logistic regression draws a line (or plane) in your data to separate one class from another.
The Core Idea
In logistic regression, a decision boundary is the invisible line (or surface) that divides your feature space into regions where the model predicts different classes. Remember the sigmoid function from the previous lesson? It maps any input to a probability between 0 and 1. The decision boundary is where this probability equals exactly 0.5—the dividing line between predicting class 0 and class 1.
Why "Linear" Boundaries?
The logistic regression model uses a linear combination of features (like w₁x₁ + w₂x₂ + b) before applying the sigmoid. This means the decision boundary itself is always a straight line in 2D, or a flat plane in higher dimensions—it's linear.
Think of it like drawing a straight line through a scatter plot of red and blue points. Points on one side of the line are predicted as "red" (class 1), and points on the other side as "blue" (class 0). The line itself represents the 50-50 threshold.
The Math Behind the Boundary
The boundary occurs where the model's linear part equals zero: w₁x₁ + w₂x₂ + b = 0. At this exact line, sigmoid(0) = 0.5. Cross the line in one direction, and the probability climbs above 0.5 (predicting class 1). Cross the other way, it drops below 0.5 (predicting class 0).
Key Takeaway: Logistic regression creates linear decision boundaries by finding the straight line (or hyperplane) that best separates classes based on where the probability threshold (typically 0.5) divides the feature space.