Network Latency and Bandwidth Constraints
What you'll learn: How variable network delays and limited bandwidth shape the design and performance of distributed systems.
The Core Challenge
When building distributed systems, you're sending data across networks—between data centers, cloud regions, or continents. Two physical realities fundamentally constrain what's possible: latency (how long it takes for a message to travel) and bandwidth (how much data can flow per second).
Latency is your round-trip delay. Even at the speed of light, a message from New York to Singapore takes ~180ms round-trip. Add in routers, switches, and processing time, and real-world latencies often reach 200-300ms. This delay is unavoidable and variable—some requests are fast, others slower.
Bandwidth is your "pipe width." A 1 Gbps connection can theoretically transfer 125 MB/s, but actual throughput depends on network congestion, packet loss, and protocol overhead. Unlike latency, bandwidth can sometimes be increased by upgrading infrastructure, but it's never unlimited.
Design Implications
These constraints force critical tradeoffs:
- Chattiness hurts: If your system requires 10 sequential network calls to complete one operation, you've multiplied latency by 10. Distributed systems favor batching and reducing round-trips.
- Data size matters: Sending 1 KB vs 10 MB has minimal latency difference but massive bandwidth impact. Large payloads can saturate links and delay other traffic.
- Variability is real: The network partition scenario you learned earlier often stems from latency spikes that cause timeouts, not just hard failures.
Unlike local function calls (nanoseconds), network calls are 100,000× slower and unpredictable. Successful distributed designs acknowledge this by caching aggressively, replicating data closer to users, and building tolerance for delays.
Key Takeaway: Network latency is unavoidable and variable; bandwidth is limited and shared. Distributed system design must minimize round-trips, manage payload sizes carefully, and assume delays will vary unpredictably.