Watermarks and Late Data
What you'll learn: How to handle out-of-order and late-arriving events in streaming systems using watermark strategies.
In streaming pipelines, data doesn't always arrive in perfect chronological order. A mobile app might buffer events offline, a network might delay packets, or clocks might be slightly out of sync across distributed systems. Watermarks are the mechanism streaming systems use to track progress through event time and decide when it's safe to consider a time window "complete."
How Watermarks Work
Think of a watermark as a moving line on a timeline that says: "I'm reasonably confident all events with timestamps before this point have arrived." It's not a guarantee—it's a heuristic. The watermark advances based on the timestamps of incoming events, typically lagging behind by some configurable amount (like 5 minutes).
When the watermark passes the end of a window (say, a 1-hour window ending at 2:00 PM), the system triggers aggregations and emits results, even if a few stragglers might still be in transit.
Handling Late Data
Late data arrives after the watermark has already passed. You have several strategies:
- Drop it: Simplest approach, acceptable for non-critical metrics
- Update results: Emit corrected aggregations (allowed triggers in systems like Apache Flink/Beam)
- Side output: Route late data to a separate stream for manual inspection
- Grace period: Configure an "allowed lateness" window where late data still updates results
For example, with a 5-minute watermark delay and 10-minute allowed lateness, events up to 15 minutes late can still update window results before being discarded.
Key Takeaway: Watermarks balance timeliness and completeness in streaming—they let you process data promptly while handling inevitable late arrivals through configurable tolerance strategies.