Course contentsShow
← Machine Learning and Deep Learning
Lesson 273 of 3,5385. Support Vector MachinesFree lesson

The C Hyperparameter

Understand how the regularization parameter C controls the trade-off between margin maximization and training error.

The C Hyperparameter

What you'll learn: How the C hyperparameter in soft-margin SVMs controls the balance between maximizing the margin and minimizing training errors.

The Core Idea

In the previous lesson, you learned that soft-margin SVMs introduce slack variables to allow some misclassifications or margin violations. But how do we decide how much error to tolerate?

The C hyperparameter is your control knob for this trade-off. It determines how heavily the SVM penalizes training errors (violations of the margin or misclassifications).

Understanding C

Think of C as the "strictness level" of your classifier:

  • Large C (e.g., C = 1000): Very strict. The model heavily penalizes any training errors, trying to classify every training point correctly. This leads to a narrower margin but fewer misclassifications on training data. Risk: overfitting.

  • Small C (e.g., C = 0.01): More lenient. The model tolerates more training errors in favor of a wider margin. This prioritizes better generalization over perfect training accuracy. Risk: underfitting.

The Mathematical Connection

Recall the soft-margin SVM objective includes both margin maximization and slack penalty terms. The C parameter weights the importance of the slack (error) term. High C means "care a lot about minimizing errors," while low C means "care more about maximizing margin."

An Analogy

Imagine you're drawing a path between two groups of people. A high C is like insisting the path stays far from everyone, even if it becomes very narrow. A low C is like drawing a wide, comfortable path, accepting that a few people might be close to or even on the wrong side.

Key Takeaway: The C hyperparameter controls the bias-variance tradeoff in SVMs—higher C reduces bias but increases variance (potential overfitting), while lower C increases bias but reduces variance (better generalization).