Kernels and Filters: Terminology
What you'll learn: You'll understand the precise meaning of "kernel," "filter," and "weight matrix" in CNNs and why these terms are often used interchangeably.
The Confusion in Terminology
In convolutional neural networks, you'll encounter three terms that refer to essentially the same thing: kernel, filter, and weight matrix. This can be confusing because different papers, frameworks, and textbooks use them differently.
What Each Term Means
Kernel is the most mathematically precise term. It refers to the small matrix of learnable weights that slides across your input during convolution. For example, a 3×3 kernel contains 9 numbers that detect specific patterns.
Filter technically refers to the complete set of kernels across all input channels. If you're processing an RGB image (3 channels) with a 3×3 operation, the "filter" includes three 3×3 kernels—one per channel. However, many practitioners use "filter" and "kernel" interchangeably.
Weight matrix emphasizes that these are learnable parameters, just like weights in a fully connected layer. This term highlights that the kernel values are optimized during training via backpropagation.
The Practical Reality
In practice, especially in PyTorch and modern deep learning:
- People often say "3×3 kernel" or "3×3 filter" to mean the same thing
- Documentation might mix these terms
- What matters is understanding the concept: a small matrix of weights that slides across input to detect features
Think of it like calling a photograph a "photo," "picture," or "image"—slightly different contexts, same underlying object.
Key Takeaway: Kernel, filter, and weight matrix all refer to the learnable parameters in a convolutional layer that slide across input data to detect features; they're largely interchangeable in modern usage.