Course contentsShow
← Machine Learning and Deep Learning
Lesson 271 of 3,5385. Support Vector MachinesFree lesson

Primal Formulation of Hard-Margin SVM

Express the SVM problem in primal form with weight vector normalization and linear inequality constraints.

Primal Formulation of Hard-Margin SVM

What You'll Learn

You'll learn how to express the hard-margin SVM optimization problem mathematically using weight vector normalization and linear constraints.

The Mathematical Setup

In the previous lessons, you saw that SVM aims to maximize the margin between classes. Now we formalize this as an optimization problem.

The primal formulation is the original way to state the SVM problem before any mathematical transformations. Think of it as the "raw" version of the recipe before simplification.

The Primal Problem

For a linearly separable dataset with labels y_i ∈ {-1, +1}, we want to:

Minimize: (1/2) ||w||²

Subject to: y_i(w · x_i + b) ≥ 1 for all training samples i

Let's break this down:

  • Objective function: Minimizing ||w||² (the squared norm of the weight vector) is equivalent to maximizing the margin 2/||w||. We use the squared form because it's mathematically easier to optimize.

  • Constraints: Each inequality ensures that every point is on the correct side of the decision boundary AND at least distance 1/||w|| away from it. The clever use of y_i (which is ±1) handles both classes in one equation.

Why "Primal"?

This formulation directly optimizes the weights w and bias b that define your decision boundary. It's called "primal" because there's a related "dual" formulation (which you'll learn later) that looks at the problem from a different mathematical angle.

Key Takeaway: The primal SVM formulation is a constrained optimization problem that finds the maximum-margin hyperplane by minimizing the weight vector norm while ensuring all points satisfy margin constraints.