What is Convolution: Mathematical Definition
You'll learn: The formal mathematical definition of discrete convolution and how it combines two functions to produce a third.
The Core Idea
Convolution is a mathematical operation that takes two functions and produces a third by sliding one function over the other, multiplying overlapping values, and summing the results. In deep learning, this operation is fundamental to how Convolutional Neural Networks process images and other grid-like data.
Mathematical Definition
For discrete signals (like images), the convolution of two functions f and g is defined as:
(f * g)[n] = Σ f[m] · g[n - m]
Where:
*denotes the convolution operation (not multiplication)nis the output positionmis the summation index over all valid positionsf[m]is typically your input signal (like pixel values)g[n - m]is the kernel or filter, flipped and shifted
Understanding Through Analogy
Think of convolution as a weighted moving average. Imagine sliding a small window (the kernel) across your data. At each position, you multiply the values in your window with the corresponding data values beneath it, sum everything up, and that sum becomes one output value. Then you shift the window and repeat.
Why "Flipping" Matters
The term g[n - m] means the kernel is mathematically flipped. In practice, deep learning frameworks often implement cross-correlation (without flipping) and still call it convolution, because learnable kernels can adapt regardless of the initial orientation.
Key Takeaway: Convolution is a mathematical operation that combines two functions by sliding, multiplying, and summing—transforming input data through learned patterns (kernels) to extract features.