Local Connectivity and Weight Sharing
What you'll learn: How convolutional layers achieve massive parameter efficiency by connecting neurons locally and reusing the same weights across the image.
The Big Efficiency Trick in CNNs
Imagine you had to hire one inspector for every single square meter of a huge factory floor, each trained differently. That would be expensive and inefficient! Instead, you train one inspector well and have them walk the entire floor systematically. That's exactly what convolution does with its filters.
Local Connectivity
Unlike fully connected layers where every input connects to every output, convolutional neurons only look at small local regions (the receptive field). A neuron in a conv layer might only see a 3×3 patch of pixels, not the entire image. This dramatically reduces connections:
- A fully connected layer from a 224×224 image to 64 neurons: 3.2 million parameters
- A 3×3 conv layer with 64 filters: just 576 parameters (plus biases)
You already saw this when computing parameter counts—local connectivity is why those numbers stay manageable.
Weight Sharing
Here's the real magic: the same kernel weights are reused at every spatial position. When your 3×3 filter slides across the image (using stride), it applies the identical weights everywhere.
This makes intuitive sense: if an edge detector works in the top-left corner, it should work in the bottom-right too! The pattern doesn't change based on location.
Together, these two principles mean one small kernel can analyze an entire large image, detecting the same features everywhere while using very few parameters.
Key Takeaway: Convolution achieves parameter efficiency through local connectivity (each neuron sees only a small patch) and weight sharing (the same kernel weights scan the entire input), allowing CNNs to process large images with remarkably few learnable parameters.