Course contentsShow
← System Design
Lesson 741 of 1,91920. Data Pipelines and ETLFree lesson

Windowing in Stream Processing

Windows group streaming data into time-based or count-based buckets for aggregations and computations.

Windowing in Stream Processing

You'll learn how to group unbounded streams of data into finite chunks so you can perform meaningful aggregations and computations.

The Challenge of Infinite Streams

In stream processing, data flows continuously without a clear start or end. But how do you calculate a "sum," "average," or "count" when the data never stops arriving? You need a way to draw boundaries around portions of the stream—this is where windowing comes in.

What Are Windows?

Windows divide an infinite stream into finite, manageable buckets based on either time or count. Think of it like taking snapshots of a river: you can't measure "all the water ever," but you can measure "the water that flowed past in the last 5 minutes."

Common Window Types

Time-based windows group events by temporal boundaries:

  • Tumbling windows: Fixed, non-overlapping intervals (e.g., every 10 seconds)
  • Sliding windows: Overlapping intervals that move continuously (e.g., last 10 seconds, updated every 2 seconds)
  • Session windows: Dynamic windows based on periods of activity separated by inactivity gaps

Count-based windows group by number of events (e.g., every 100 messages).

Why Windowing Matters

Without windows, you can't answer questions like "What was the average response time in the last minute?" or "How many orders were placed this hour?" Windows make stream processing practical by converting infinite streams into finite computations that produce timely results.

Key Takeaway: Windowing transforms unbounded streams into bounded chunks based on time or count, enabling aggregations and computations that would otherwise be impossible on infinite data flows.