Course contentsShow
← Machine Learning and Deep Learning
Lesson 280 of 3,5385. Support Vector MachinesFree lesson

Common Kernel Functions

Polynomial, RBF (Gaussian), and sigmoid kernels with their mathematical forms and properties.

Common Kernel Functions

What you'll learn: You'll discover the three most popular kernel functions used in SVMs and understand their mathematical forms and when to use each one.

Core Idea

Once you understand that kernels let you transform data into higher dimensions (without explicit computation), the next question is: which kernel should I use? There are three workhorses in practice, each with unique properties that make them suited to different patterns in data.

The Big Three Kernels

1. Polynomial Kernel

Mathematical form: K(x, x') = (γ·x^T·x' + r)^d

  • γ (gamma): scaling parameter
  • r: independent term (often 0 or 1)
  • d: degree of the polynomial

Use when: Your decision boundary looks like it needs polynomial curves. A degree-2 kernel creates ellipses and parabolas; degree-3 adds more complex curves.

2. RBF (Radial Basis Function / Gaussian) Kernel

Mathematical form: K(x, x') = exp(-γ·||x - x'||²)

  • γ controls the "reach" of each training point's influence
  • Creates smooth, localized bumps in feature space

Use when: You don't know the pattern ahead of time (it's the default choice). It can approximate any continuous function and handles highly non-linear boundaries beautifully.

3. Sigmoid Kernel

Mathematical form: K(x, x') = tanh(γ·x^T·x' + r)

Mimics a neural network activation function.

Use when: Rarely used in practice; mainly for specific applications where neural-network-like behavior is desired.

Properties Summary

  • Polynomial: Adjustable complexity via degree; can overfit with high d
  • RBF: Most flexible; gamma controls smoothness
  • Sigmoid: Less common; can be unstable

Key Takeaway: The RBF kernel is your go-to default for non-linear SVMs, polynomial kernels work when you know the degree of complexity needed, and sigmoid kernels are niche players in the kernel family.