Course contentsShow
← Cloud and Infrastructure
Lesson 2600 of 2,98956. Data Pipelines and StreamingFree lesson

What is Streaming Data?

Understand streaming data concepts, characteristics, and how it differs from batch processing.

What is Streaming Data?

You'll learn: What streaming data is, its core characteristics, and how it fundamentally differs from batch processing approaches you've already used in microservices and serverless architectures.

Core Concept

Streaming data is continuous, unbounded flows of data records generated constantly over time, requiring immediate or near-real-time processing as events arrive. Unlike batch processing where you collect data, wait, then process it all at once, streaming processes each event incrementally as it flows through your system.

Think of batch processing like receiving mail once daily and sorting it all at 5pm. Streaming is like a fire hose of information constantly spraying—you must handle each drop as it arrives, not wait for the hose to stop (it never does).

Key Characteristics

Unbounded: Streaming data has no predetermined end. Unlike a batch file with a fixed number of records, streams continue indefinitely (user clicks, IoT sensor readings, logs).

Time-sensitive: Events have timestamps and often must be processed within strict time windows. Stale data loses value quickly.

Volume and velocity: High throughput (thousands to millions of events per second) demands different infrastructure than batch systems.

Event-driven: Each data record represents something that happened—a purchase, temperature reading, or API call—connecting naturally to event-driven architectures you've already studied.

Streaming vs Batch

Batch processing reads complete datasets, processes them, and writes results. Streaming processes events individually or in small micro-batches with low latency. Batch optimizes for throughput; streaming optimizes for latency and responsiveness.

Key Takeaway: Streaming data is continuous, time-sensitive, and unbounded, requiring architectures that process events incrementally as they arrive rather than waiting to accumulate complete datasets like batch processing.