Course contentsShow
← Computer Science
Lesson 1151 of 2,87227. Pipelining and PerformanceFree lesson

Pipeline Throughput and Latency

Distinguish between instruction throughput (instructions per cycle) and latency (cycles per instruction) in pipelines.

Pipeline Throughput and Latency

What You'll Learn

You'll learn the crucial difference between how fast a pipeline completes individual instructions versus how many instructions it can process simultaneously.

The Two Faces of Pipeline Performance

When evaluating a pipelined CPU (like the five-stage pipeline you've just learned), we measure success in two very different ways:

Latency: Time Per Instruction

Latency measures how long it takes for a single instruction to travel through the entire pipeline from start to finish. It's measured in cycles per instruction (CPI).

In our classic five-stage pipeline (Fetch, Decode, Execute, Memory, Write-Back), one instruction takes 5 cycles to complete all stages. Even though stages overlap, that instruction doesn't finish until it exits the final stage.

Analogy: Think of a car wash with five stations. Your car takes 25 minutes to go through all five (5 minutes each). That's latency — the total time your specific car spends getting washed.

Throughput: Instructions Per Time

Throughput measures how many instructions the pipeline can complete per cycle once it's full. It's measured in instructions per cycle (IPC).

In an ideal pipeline, once filled, we complete one instruction per cycle. While instruction A finishes Write-Back, instruction B finishes Memory, C finishes Execute, and so on. Every cycle, one instruction exits!

Car wash analogy continued: Once all five stations are busy, a clean car exits every 5 minutes, even though each individual car takes 25 minutes. That's throughput — the rate of completion.

Why Both Matter

  • Latency affects responsiveness for individual tasks
  • Throughput determines overall program execution speed

A pipeline doesn't make single instructions faster (latency stays the same), but it dramatically improves throughput by overlapping work.

Key Takeaway: Pipeline latency measures cycles per instruction (how long each takes), while throughput measures instructions per cycle (completion rate). Pipelining improves throughput without reducing latency.