Course contentsShow
← System Design
Lesson 1780 of 1,91942. Real-World System: Search and TypeaheadFree lesson

Distributed Query Coordination

Coordinating scatter-gather queries across shards and merging results while handling timeouts and failures.

Distributed Query Coordination

What You'll Learn

You'll learn how to orchestrate parallel queries across multiple index shards and merge their results while gracefully handling timeouts and failures.

The Scatter-Gather Pattern

When your search index is horizontally partitioned across multiple shards (as covered in your earlier lessons), a single user query must fan out to all relevant shards simultaneously, then merge their results into a unified response.

Think of it like asking librarians in different library branches to each find the top 10 books matching "quantum physics." You wait for all responses, then combine and re-rank them to produce the final top 10.

Core Coordination Responsibilities

The query coordinator (or aggregator node) handles:

  1. Query Broadcasting: Send the identical search request to all shards in parallel
  2. Partial Result Collection: Each shard returns its local top-K results with scores
  3. Global Merge-Sort: Combine results from all shards, re-rank by score, and return the global top-K
  4. Timeout Management: Set aggressive per-shard timeouts (e.g., 100-200ms) to maintain p99 latency
  5. Partial Failure Handling: If one shard times out or fails, decide whether to return incomplete results or fail the query

Handling Failures Gracefully

Timeout strategy: Use circuit breakers per shard. If shard-3 repeatedly times out, temporarily exclude it and alert operators.

Degraded mode: Return results from N-1 shards with a warning flag rather than failing completely—users get 95% coverage instead of nothing.

Retry logic: For critical queries, retry failed shards once with a shorter timeout, but never block the response.

Monitoring Essentials

Track per-shard latency percentiles, timeout rates, and result contribution to detect hot shards or failing replicas early.

Key Takeaway: Distributed query coordination transforms multiple partial results into a unified response through parallel scatter-gather execution, aggressive timeouts, and graceful degradation when shards fail.