Course contentsShow
← Machine Learning and Deep Learning
Lesson 282 of 3,5385. Support Vector MachinesFree lesson

RBF Kernel and Gamma Parameter

Deep dive into the radial basis function kernel and its gamma hyperparameter's influence on decision boundaries.

RBF Kernel and Gamma Parameter

What you'll learn: How the Radial Basis Function (RBF) kernel creates curved decision boundaries and how the gamma parameter controls their flexibility.

What is the RBF Kernel?

The Radial Basis Function (RBF) kernel is the most popular kernel for SVMs. It measures similarity between data points based on their distance in the original feature space. The formula looks like:

K(x, x') = exp(-γ ||x - x'||²)

Think of it as a "proximity detector": points that are close together get high similarity scores (near 1), while distant points get low scores (near 0). The exponential decay creates smooth, circular influence zones around each support vector.

Understanding the Gamma Parameter

Gamma (γ) controls how far the influence of a single training example reaches:

  • Low gamma (e.g., 0.01): Each point influences a wide area → smooth, simple decision boundaries → higher bias, lower variance
  • High gamma (e.g., 10): Each point influences only nearby points → complex, wiggly decision boundaries → lower bias, higher variance

Analogy

Imagine gamma as the "reach" of a flashlight in fog:

  • Low gamma = bright floodlight illuminating broadly (generalizes widely)
  • High gamma = narrow laser beam (focuses tightly on nearby points)

The Practical Impact

With low gamma, your SVM creates gentle curves that might underfit. With high gamma, it can create island-like decision regions around individual points, risking overfitting. The gamma parameter essentially controls the bias-variance tradeoff in kernel SVMs.

Key Takeaway: The RBF kernel's gamma parameter determines how "locally" each training example influences the decision boundary—lower values create smoother boundaries, while higher values allow tighter fits around individual points.