Course contentsShow
← Machine Learning and Deep Learning
Lesson 269 of 3,5385. Support Vector MachinesFree lesson

Hard-Margin SVM Objective

Formulate the hard-margin SVM optimization problem: maximizing margin subject to perfect separation constraints.

Hard-Margin SVM Objective

What You'll Learn

You'll learn how to formulate the optimization problem that finds the best separating line (or hyperplane) with the widest possible margin between classes.

The Core Idea

Recall from the previous lesson that the margin is the distance from the decision boundary to the nearest data point. A Support Vector Machine (SVM) wants to find the boundary that maximizes this margin—creating the widest "safety buffer" between classes.

The hard-margin SVM assumes perfect linear separability: every training example can be correctly classified with no errors. This leads to a clear mathematical goal:

Find the decision boundary that maximizes the margin while correctly classifying all training points.

The Optimization Problem

Mathematically, we want to:

Maximize: The margin width (distance between the boundary and nearest points)

Subject to: Every training example must be on the correct side of the boundary, with at least the margin distance away.

For a decision boundary defined by weights w and bias b, this becomes:

  • Objective: Maximize the margin (which relates to minimizing ||w||)
  • Constraints: For each training point (xᵢ, yᵢ), ensure yᵢ(w·xᵢ + b) ≥ 1

The constraint says: "positive examples must be at least +1 on the positive side, and negative examples must be at least -1 on the negative side."

The Analogy

Think of building a fence between two properties. You want the widest possible neutral zone (margin) where neither neighbor can build, while ensuring all of property A's structures stay fully on side A, and all of property B's structures stay on side B.

Key Takeaway: The hard-margin SVM objective is a constrained optimization problem that maximizes margin while requiring perfect separation of all training examples—finding the "safest" possible decision boundary.