Evaluating Throughput vs Latency

Make architectural trade-offs between maximizing total throughput and reducing single-request latency.