Common Kernel Functions
What you'll learn: You'll discover the three most popular kernel functions used in SVMs and understand their mathematical forms and when to use each one.
Core Idea
Once you understand that kernels let you transform data into higher dimensions (without explicit computation), the next question is: which kernel should I use? There are three workhorses in practice, each with unique properties that make them suited to different patterns in data.
The Big Three Kernels
1. Polynomial Kernel
Mathematical form: K(x, x') = (γ·x^T·x' + r)^d
- γ (gamma): scaling parameter
- r: independent term (often 0 or 1)
- d: degree of the polynomial
Use when: Your decision boundary looks like it needs polynomial curves. A degree-2 kernel creates ellipses and parabolas; degree-3 adds more complex curves.
2. RBF (Radial Basis Function / Gaussian) Kernel
Mathematical form: K(x, x') = exp(-γ·||x - x'||²)
- γ controls the "reach" of each training point's influence
- Creates smooth, localized bumps in feature space
Use when: You don't know the pattern ahead of time (it's the default choice). It can approximate any continuous function and handles highly non-linear boundaries beautifully.
3. Sigmoid Kernel
Mathematical form: K(x, x') = tanh(γ·x^T·x' + r)
Mimics a neural network activation function.
Use when: Rarely used in practice; mainly for specific applications where neural-network-like behavior is desired.
Properties Summary
- Polynomial: Adjustable complexity via degree; can overfit with high d
- RBF: Most flexible; gamma controls smoothness
- Sigmoid: Less common; can be unstable
Key Takeaway: The RBF kernel is your go-to default for non-linear SVMs, polynomial kernels work when you know the degree of complexity needed, and sigmoid kernels are niche players in the kernel family.