Course contentsShow
← Machine Learning and Deep Learning
Lesson 278 of 3,5385. Support Vector MachinesFree lesson

Feature Space Transformations

Mapping input features to higher-dimensional spaces where linear separation becomes possible.

Feature Space Transformations

What you'll learn: How transforming your input features into a higher-dimensional space can turn impossible-to-separate data into linearly separable data.

The Core Problem

Sometimes your data isn't linearly separable in its original form. Imagine trying to draw a straight line to separate two classes arranged in concentric circles—it's impossible. But what if you could lift those points into a higher-dimensional space where suddenly a flat plane could separate them?

What is a Feature Space Transformation?

A feature space transformation (also called a feature mapping) takes your original input features and maps them to a new, usually higher-dimensional space. In this new space, patterns that were tangled together become cleanly separable by a linear boundary.

Simple example: Given a 1D input x, you might map it to 2D as [x, x²]. Data points that couldn't be separated by a single threshold in 1D might now be separable by a line in 2D.

Why This Matters for SVMs

Linear SVMs can only draw straight decision boundaries. When your data isn't linearly separable, you have two choices:

  1. Give up on linear methods
  2. Transform the features so they become linearly separable

By mapping features from their original space (say, 2D) to a richer space (maybe 5D or even infinite dimensions), you create new "views" of your data where linear separation is possible. After finding the linear boundary in the transformed space, predictions automatically work in the original space.

Key Takeaway: Feature space transformations let linear classifiers solve nonlinear problems by creating higher-dimensional representations where linear boundaries become sufficient—the foundation for kernel methods in SVMs.