Non-linear Decision Boundaries via Feature Engineering
What You'll Learn
You'll discover how to make logistic regression flexible enough to solve problems where classes can't be separated by a straight line—by cleverly transforming your input features.
The Core Problem
Logistic regression naturally creates linear decision boundaries—straight lines (or flat hyperplanes in higher dimensions). But what if your data looks like two concentric circles, or a spiral pattern? A straight line can't separate those classes!
The Solution: Engineer Better Features
Instead of changing the algorithm, we transform the input features to make the problem linearly separable in a new space.
Two Key Transformations
Polynomial Features: Create powers of existing features.
- From
[x₁, x₂]→[x₁, x₂, x₁², x₁x₂, x₂²] - Now logistic regression can fit curved boundaries like circles or parabolas
Interaction Features: Multiply features together.
- Captures relationships like "the effect of feature 1 depends on feature 2"
- Already included in polynomial features, but can be added independently
How It Works
Imagine trying to separate red and blue dots arranged in two circles. In the original (x, y) space, no straight line works. But if you add a feature z = x² + y² (distance from origin), the circles become separated by a simple threshold on z—a linear boundary in the new 3D space!
You already learned about creating these features in lesson 206. Now you understand why: they let logistic regression model complex, curved decision boundaries while still using the same gradient descent and cross-entropy loss you've already mastered.
Key Takeaway: Logistic regression is linear in its features, but by engineering polynomial and interaction features, you can model arbitrarily complex non-linear decision boundaries without changing the core algorithm.