Course contentsShow
← Machine Learning and Deep Learning
Lesson 1369 of 3,53830. Vision TransformersFree lesson

Conditional DETR and Query Improvements

Using conditional spatial queries and reference points to accelerate training convergence.

Conditional DETR and Query Improvements

What you'll learn: How Conditional DETR uses conditional spatial queries and reference points to make detection transformers train much faster.

The Convergence Problem in DETR

Remember how DETR uses learnable object queries that start as random vectors? The original DETR architecture requires hundreds of epochs to converge because each query must independently learn where to look and what to detect. This is like asking 100 people to find items in a dark room without telling them where to start searching—they'll eventually succeed, but it takes a very long time.

Conditional Spatial Queries

Conditional DETR solves this by giving each query a conditional reference point early in training. Instead of queries being pure content vectors, each query maintains an explicit spatial reference (an x, y coordinate in the image) that guides where it should focus attention.

Think of it as giving each searcher a flashlight that points to a specific area—they still need to learn what object is there, but they already know roughly where to look.

How It Works

  1. Each object query produces a reference point (2D coordinates)
  2. Cross-attention keys and values are spatially modulated based on distance from this reference point
  3. The query attends more strongly to image features near its reference point
  4. Reference points are updated across decoder layers, allowing queries to refine their search region

This conditional spatial guidance dramatically reduces the search space, cutting training time from 500 epochs to ~50 epochs while maintaining or improving accuracy.

Key Takeaway: Conditional DETR accelerates training by giving each object query an explicit spatial reference point, letting it focus attention on relevant image regions rather than searching the entire feature map blindly.