Course contentsShow
← Machine Learning and Deep Learning
Lesson 1366 of 3,53830. Vision TransformersFree lesson

Object Queries and Learned Positional Embeddings

Understanding object queries as learned slots that attend to image features to predict boxes.

Object Queries and Learned Positional Embeddings

What you'll learn: How DETR uses learnable "object queries" as placeholders that discover and attend to objects in an image to predict bounding boxes.

What Are Object Queries?

In the Detection Transformer (DETR), object queries are learnable embeddings that act as "slots" or "question marks" for potential objects in an image. Think of them as N blank forms (typically N=100) where each form will be filled with information about one detected object—or remain empty if fewer than N objects exist.

Unlike traditional detectors that scan sliding windows or anchor boxes across the image, DETR initializes these queries as learned parameters during training. Each query learns to specialize in finding objects at different positions, scales, or categories through the transformer's attention mechanism.

How They Work

The decoder in DETR takes these object queries and uses cross-attention to look at the encoded image features. Each query essentially asks: "Is there an object here that matches what I'm looking for?" Through attention, queries gather relevant visual information from the image and output:

  • A class prediction (what object, or "no object")
  • A bounding box (where it is)

The queries themselves have learned positional embeddings added to them, helping differentiate the N slots from each other. These aren't spatial positions in the image—they're abstract identifiers that help each query develop a unique "personality" for finding objects.

Why This Matters

This approach eliminates hand-crafted components like anchor boxes and Non-Maximum Suppression. The bipartite matching (Hungarian algorithm) ensures each query matches to at most one ground-truth object during training, forcing queries to learn complementary detection strategies.

Key Takeaway: Object queries are learnable embeddings that serve as specialized detectors—each query attends to image features through transformers to discover and localize objects, replacing traditional sliding-window approaches with learned, parallel prediction slots.