Decision Boundaries and Separability
What you'll learn: How classification models draw invisible lines to separate different classes of data, and when that's easy versus hard.
The Invisible Dividing Line
When a classification model makes predictions, it creates an invisible decision boundary — a line (or curve, or surface) that separates one class from another in feature space. Think of it like drawing a fence between cats and dogs in a yard: everything on one side gets labeled "cat," everything on the other side gets labeled "dog."
For data with two features, this boundary appears as a literal line or curve on a 2D plot. For three features, it's a surface in 3D space. Beyond that, it exists mathematically even though we can't visualize it.
Linear vs Nonlinear Boundaries
A linear decision boundary is a straight line (or flat plane in higher dimensions). Simple models like basic logistic regression (which you'll learn soon) create these straight boundaries — they can only say "everything to the left is Class A, everything to the right is Class B."
A nonlinear decision boundary is curved or wiggly. More complex models can create these flexible boundaries to wrap around oddly-shaped clusters of data.
Separability: Can You Draw the Line?
Linearly separable data means you can draw a straight line that perfectly separates the classes. Imagine plotting exam scores: if all students who studied >5 hours passed and all who studied <5 hours failed, a vertical line at 5 hours cleanly divides them.
Non-separable data means no straight line works — the classes overlap or interweave. Maybe some students who studied 6 hours still failed, and some who studied 3 hours passed. Now you need a curved boundary or must accept some errors.
Key Takeaway: Decision boundaries are the dividing lines classification models create between classes; linearly separable data can be split by a straight line, while non-separable data requires curved boundaries or tolerating some misclassifications.