Course contentsShow
← Machine Learning and Deep Learning
Lesson 971 of 3,53822. Computer Vision TasksFree lesson

DETR: Detection with Transformers

End-to-end detection using transformer encoder-decoder without hand-crafted anchors or NMS.

DETR: Detection with Transformers

You'll learn how transformers revolutionized object detection by eliminating hand-crafted components like anchors and Non-Maximum Suppression.

The Breakthrough Idea

Until now, all the object detectors you've seen (YOLO, SSD, RetinaNet) relied on carefully engineered components: anchor boxes to propose regions and Non-Maximum Suppression (NMS) to filter duplicate predictions. DETR (DEtection TRansformer) throws these away entirely, treating object detection as a direct set prediction problem.

Think of traditional detectors as artists who sketch many rough drafts (anchors) then erase duplicates (NMS). DETR is like an artist who directly paints exactly the objects they see—no drafts, no erasing.

How DETR Works

DETR uses a transformer encoder-decoder architecture borrowed from natural language processing:

  1. CNN Backbone: Extracts image features (like ResNet-50)
  2. Transformer Encoder: Processes these spatial features with self-attention, learning relationships between different image regions
  3. Transformer Decoder: Takes learned queries (think of these as "slots" for objects) and predicts a fixed number of objects directly
  4. Set Prediction: Outputs exactly N predictions (e.g., 100), where each contains a class label and bounding box

The magic? DETR uses bipartite matching during training to assign each ground-truth object to exactly one prediction, eliminating the need for NMS. Empty slots predict "no object."

This end-to-end approach is elegant: you feed an image in, and you get detections out—no post-processing tricks needed.

Key Takeaway: DETR simplifies object detection by treating it as direct set prediction using transformers, eliminating hand-crafted components like anchors and NMS for a truly end-to-end learnable system.