Course contentsShow
← Machine Learning and Deep Learning
Lesson 256 of 3,5384. Classification with Logistic RegressionFree lesson

Non-linear Decision Boundaries via Feature Engineering

Creating polynomial and interaction features to enable logistic regression to model non-linear boundaries.

Non-linear Decision Boundaries via Feature Engineering

What You'll Learn

You'll discover how to make logistic regression flexible enough to solve problems where classes can't be separated by a straight line—by cleverly transforming your input features.

The Core Problem

Logistic regression naturally creates linear decision boundaries—straight lines (or flat hyperplanes in higher dimensions). But what if your data looks like two concentric circles, or a spiral pattern? A straight line can't separate those classes!

The Solution: Engineer Better Features

Instead of changing the algorithm, we transform the input features to make the problem linearly separable in a new space.

Two Key Transformations

Polynomial Features: Create powers of existing features.

  • From [x₁, x₂] → [x₁, x₂, x₁², x₁x₂, x₂²]
  • Now logistic regression can fit curved boundaries like circles or parabolas

Interaction Features: Multiply features together.

  • Captures relationships like "the effect of feature 1 depends on feature 2"
  • Already included in polynomial features, but can be added independently

How It Works

Imagine trying to separate red and blue dots arranged in two circles. In the original (x, y) space, no straight line works. But if you add a feature z = x² + y² (distance from origin), the circles become separated by a simple threshold on z—a linear boundary in the new 3D space!

You already learned about creating these features in lesson 206. Now you understand why: they let logistic regression model complex, curved decision boundaries while still using the same gradient descent and cross-entropy loss you've already mastered.

Key Takeaway: Logistic regression is linear in its features, but by engineering polynomial and interaction features, you can model arbitrarily complex non-linear decision boundaries without changing the core algorithm.