Calculating Pipeline Throughput and Latency

Analyze how pipelining increases instruction throughput by reducing the clock cycle time to the duration of the slowest stage.

In this lesson, you will learn how pipelining improves processor performance by allowing multiple instructions to overlap, effectively decoupling throughput from individual instruction latency.

The Pipeline Concept

In a non-pipelined (single-cycle) processor, the entire execution of an instruction—from fetching to writing back to a register—must complete within one clock cycle. This means the clock period is determined by the "slowest" possible instruction. Pipelining changes this by breaking the instruction execution into smaller, independent stages (like Fetch, Decode, Execute, Memory, and Write-Back).

Throughput vs. Latency

To understand the benefit, we must distinguish between two metrics:

Latency: The time it takes for one single instruction to travel from the start of the pipeline to the very end. Pipelining actually increases the total latency slightly due to the overhead of pipeline registers between stages.
Throughput: The rate at which instructions are completed. In a perfect pipeline, a new instruction finishes every single clock cycle, even though each individual instruction takes five cycles to complete.

The Clock Cycle Constraint

The "secret" to high throughput is that the clock cycle time is no longer dictated by the sum of all stages. Instead, the clock cycle time is determined by the duration of the slowest stage in the pipeline.

Imagine a 5-stage pipeline where the stages take varying times:

Fetch: 200ps
Decode: 150ps
Execute: 250ps
Memory: 200ps
Write-Back: 150ps

In a non-pipelined system, the cycle time would be $200+150+250+200+150 = 950ps$ . In our pipelined system, the clock must only be long enough to accommodate the slowest stage: 250ps.

Even though it takes $5 \times 250ps = 1250ps$ to finish a single instruction (the latency), the processor finishes one instruction every 250ps (the throughput). By balancing the stages so they are as equal in time as possible, we maximize the frequency at which the CPU can "pump" instructions through the system.

Key Takeaway: Pipelining increases throughput by shortening the clock cycle to match the duration of the single slowest stage, allowing the processor to overlap the execution of multiple instructions simultaneously.