Course contentsShow
← Machine Learning and Deep Learning
Lesson 238 of 3,5384. Classification with Logistic RegressionFree lesson

Decision Boundaries and Separability

Understanding linear and nonlinear decision boundaries, and the concept of linearly separable vs. non-separable data.

Decision Boundaries and Separability

What you'll learn: How classification models draw invisible lines to separate different classes of data, and when that's easy versus hard.

The Invisible Dividing Line

When a classification model makes predictions, it creates an invisible decision boundary — a line (or curve, or surface) that separates one class from another in feature space. Think of it like drawing a fence between cats and dogs in a yard: everything on one side gets labeled "cat," everything on the other side gets labeled "dog."

For data with two features, this boundary appears as a literal line or curve on a 2D plot. For three features, it's a surface in 3D space. Beyond that, it exists mathematically even though we can't visualize it.

Linear vs Nonlinear Boundaries

A linear decision boundary is a straight line (or flat plane in higher dimensions). Simple models like basic logistic regression (which you'll learn soon) create these straight boundaries — they can only say "everything to the left is Class A, everything to the right is Class B."

A nonlinear decision boundary is curved or wiggly. More complex models can create these flexible boundaries to wrap around oddly-shaped clusters of data.

Separability: Can You Draw the Line?

Linearly separable data means you can draw a straight line that perfectly separates the classes. Imagine plotting exam scores: if all students who studied >5 hours passed and all who studied <5 hours failed, a vertical line at 5 hours cleanly divides them.

Non-separable data means no straight line works — the classes overlap or interweave. Maybe some students who studied 6 hours still failed, and some who studied 3 hours passed. Now you need a curved boundary or must accept some errors.

Key Takeaway: Decision boundaries are the dividing lines classification models create between classes; linearly separable data can be split by a straight line, while non-separable data requires curved boundaries or tolerating some misclassifications.