Course contentsShow
← Software Engineering
Lesson 1387 of 1,93341. Scalability and Distributed SystemsFree lesson

Measuring Scalability: Throughput and Latency

Define key metrics for scalability testing and understand the relationship between load, throughput, and latency.

Measuring Scalability: Throughput and Latency

You'll learn how to quantify system scalability using throughput and latency metrics and understand their relationship with load.

The Core Metrics

When you scale a system, you need concrete numbers to tell if it's working. The two fundamental metrics are:

Throughput measures how much work your system completes in a given time period—think requests per second, transactions per minute, or messages processed per hour. It's the "volume" capacity of your system.

Latency measures how long individual operations take—the time from request submission to response receipt. It's the "speed" experience for each user.

The Load-Performance Relationship

As you increase load (concurrent users, incoming requests), you'll observe:

  • Ideal scenario: Throughput increases linearly while latency stays constant
  • Reality: Throughput increases but eventually plateaus; latency starts rising as resources become saturated
  • Overload: Throughput may drop while latency skyrockets

Think of a highway: throughput is cars per hour, latency is travel time per car. Light traffic? Both are great. Rush hour? The highway moves many cars (good throughput) but each trip takes longer (worse latency). Gridlock? Nobody moves (both metrics collapse).

Why Both Matter

You can't optimize for just one. High throughput with terrible latency means users wait forever. Low latency with poor throughput means your system can't handle realistic load. Scalability testing reveals how these metrics degrade under increasing load—and where your bottlenecks live.

Track percentiles too: the 95th or 99th percentile latency shows your worst-case user experience, not just the average.

Key Takeaway: Throughput measures system capacity (work per time), latency measures user experience (time per operation), and both degrade as load increases—successful scaling maintains acceptable levels of both under realistic load.