In this lesson, you’ll learn how to distinguish between throughput and latency and why optimizing for one often impacts the other.
When tuning your Node.js server, you must decide what "performance" means for your specific use case. These two metrics are often confused, but they measure fundamentally different behaviors:
- Latency is the time it takes for a single request to travel from the client to the server and back (round-trip time). Users care about this when they click a button and expect an immediate UI update.
- Throughput is the total number of requests your server can handle per second (RPS). System architects care about this when they need to ensure the server doesn't buckle under heavy, concurrent traffic.
The Trade-off
Think of a grocery store checkout. Latency is how fast one customer gets through the line. Throughput is how many customers finish their shopping in an hour. If you hire a super-fast cashier, you reduce latency. If you open ten checkout lanes, you increase throughput.
In Node.js, you can use benchmarking tools like autocannon to simulate these scenarios. A server optimized for low latency might keep logic simple and avoid expensive computations, while a server optimized for high throughput might use Payload Compression Middleware or Zero-Copy File Serving to handle more concurrent connections without saturating the event loop.
Practical Measurement
If you run a benchmark, pay attention to the distribution:
// A simple way to measure latency in your logs
const start = process.hrtime();
// ... process request ...
const diff = process.hrtime(start);
console.log(`Latency: ${diff[0] * 1e3 + diff[1] / 1e6}ms`);
If you notice your latency spikes as you increase concurrent connections, your throughput has hit a bottleneck (likely Memory Management or Epoll/Kqueue limits). Conversely, if your latency is low but your total requests per second are also low, you might have idle resources that aren't being fully utilized.
Key Takeaway: Low latency ensures a snappy experience for the individual user, while high throughput ensures your server can scale to support a massive crowd.