Course contentsShow
← Software Engineering
Lesson 1207 of 1,93336. Performance EngineeringFree lesson

Request Rate and Concurrency

Understanding requests per second, concurrent users, and how they relate to system capacity and behavior.

Request Rate and Concurrency

You'll learn how request rate and concurrent users reveal different facets of system load and capacity.

What Are Request Rate and Concurrency?

Request rate measures how many requests arrive per unit of time—typically expressed as requests per second (RPS) or queries per second (QPS). It tells you the speed at which work enters your system.

Concurrency measures how many requests are being processed simultaneously at any given moment. It tells you how much work is in-flight within your system at once.

These metrics are related but distinct, and understanding both is crucial for capacity planning and performance analysis.

The Relationship

Think of a restaurant: request rate is how many customers walk through the door per hour, while concurrency is how many are seated and being served at any moment. A restaurant might serve 60 customers per hour (1 per minute) but only have 15 concurrent diners if each meal takes 15 minutes.

The relationship follows: Concurrency ≈ Request Rate × Average Response Time (this is Little's Law). If your system handles 100 RPS with an average response time of 50ms, you'd expect roughly 5 concurrent requests at any moment.

Why Both Matter

High request rate with low concurrency suggests your system processes requests quickly. High concurrency with moderate request rate suggests slow processing or blocked threads. Understanding this distinction helps diagnose whether you need faster processing, more parallelism, or better queueing strategies.

Monitoring both metrics reveals system behavior under load: spiking concurrency without rising request rate often signals a performance degradation causing requests to pile up.

Key Takeaway: Request rate measures arrival speed; concurrency measures simultaneous work in-flight. Together, they define system load and reveal capacity constraints.