Course contentsShow
← Machine Learning and Deep Learning
Lesson 1363 of 3,53830. Vision TransformersFree lesson

Limitations of CNN-Based Object Detection

Why traditional CNN detectors struggle with long-range dependencies and how Transformers address these issues.

Limitations of CNN-Based Object Detection

What You'll Learn

You'll understand why traditional CNN-based object detectors struggle with capturing long-range relationships between image regions and how this affects detection performance.

The Core Problem: Local Vision

CNNs are built on convolution operations that examine small, local patches of an image at a time. While this local processing is computationally efficient and excellent at detecting edges, textures, and nearby patterns, it creates a fundamental limitation: limited receptive fields.

Think of it like looking through a magnifying glass—you see details clearly in one spot, but lose the "big picture" context. To understand broader relationships, CNNs must stack many layers, gradually expanding what each neuron "sees." But this creates three major issues:

1. Long-Range Dependencies Are Hard

Objects often relate to distant parts of an image. A baseball glove makes more sense when you see the distant baseball, or a person's action becomes clearer when you see what they're looking at across the scene. CNNs struggle to connect these distant relationships efficiently.

2. Fixed Hierarchical Structure

CNNs use rigid, pyramidal feature hierarchies—you can't easily "look back" at earlier layers' information without complex skip connections or feature pyramid networks. This makes it hard to dynamically integrate multi-scale context.

3. Computational Scaling

To increase the receptive field enough to capture image-wide relationships, you need very deep networks or large kernels, which become computationally expensive and difficult to train.

Transformers address these limitations through self-attention mechanisms that let every image patch directly "attend to" every other patch in a single operation, capturing global context immediately without deep stacking.

Key Takeaway: CNNs excel at local patterns but struggle with long-range dependencies due to limited receptive fields, making Transformers with global self-attention a powerful alternative for tasks requiring whole-image understanding.