Course contentsShow
← System Design
Lesson 1479 of 1,91936. Data Partitioning Strategies in DepthFree lesson

Directory Service as Single Point of Failure

Addressing availability, consistency, and caching challenges in directory lookups.

Directory Service as Single Point of Failure

What you'll learn: How the directory service's central role creates availability, consistency, and caching challenges in partitioned systems.

The Central Vulnerability

In directory partitioning, the directory service acts as a lookup table mapping partition keys to physical servers. Every read and write must consult this directory to find the right partition. This creates a single point of failure — if the directory goes down, the entire system becomes unreachable, even though the data partitions themselves are healthy.

Think of it like a library catalog system: even if all the books are on the shelves, you can't find anything if the catalog computer crashes.

Three Critical Challenges

1. Availability Challenge

The directory must be highly available since every operation depends on it. A brief outage cascades to complete system unavailability. Solutions involve replicating the directory itself, but this introduces new complexity.

2. Consistency Challenge

When the directory changes (partitions move, split, or merge), all replicas must reflect those changes. Stale directory information routes requests to wrong servers, causing failures. Strong consistency is often required, which conflicts with availability goals.

3. Caching Challenge

To reduce directory lookup latency, clients and servers cache mappings. But cached entries can become stale after partition changes. Aggressive caching improves performance but increases the window for routing errors. Cache invalidation across distributed clients is notoriously difficult.

The Fundamental Tradeoff

You must balance directory availability (replicate widely, cache aggressively) against consistency (ensure all replicas and caches reflect current state). Over-replicating creates consistency nightmares; under-replicating creates availability bottlenecks.

Key Takeaway: The directory service's centrality makes it a critical bottleneck requiring careful tradeoffs between availability (replication and caching) and consistency (accurate, up-to-date routing information) — often more complex than managing the data partitions themselves.