In this lesson, you will learn how to use manual savepoints to capture a consistent state of your streaming application, enabling safe version upgrades and cluster migrations.
What is a Manual Savepoint?
While automated checkpoints are managed by the system to handle unexpected failures, manual savepoints are user-triggered snapshots of your stream processing job’s state. Think of a savepoint as a "frozen moment" of your application. It records exactly what the stream was doing—including all aggregate counts, window buffers, and watermark positions—at a specific point in time. Because the state is stored in durable external storage, you can stop a job, perform a major logic update, and restart it from the exact state where you left off.
Why Use Savepoints for Versioning?
When you upgrade your application logic (for instance, changing a stateful aggregation function) or migrate your cluster to new infrastructure, you risk losing the state integrity required for accurate downstream reporting. A savepoint decouples your data state from the lifecycle of a specific deployment.
By triggering a savepoint, you create a baseline. You can then shut down your current job, update your code or cluster configuration, and deploy the new version by specifying the path to that saved state. The engine maps the old state to the new topology, ensuring that your stateful operations—like MapGroupsWithState or window aggregations—remain mathematically consistent with the data processed before the update.
Execution Pattern
Executing a savepoint is typically done via the command-line interface provided by your stream processing engine. A standard workflow looks like this:
# Trigger the savepoint to a specific directory
bin/flink savepoint <job_id> /tmp/savepoints
# Stop the job and perform the migration/update
bin/flink stop <job_id>
# Redeploy from the manual savepoint
bin/flink run -s /tmp/savepoints/<savepoint_path> my-new-application.jar
Using this approach, you ensure that the internal state—the "memory" of your streaming pipeline—is successfully transitioned into the new environment without forcing a full re-process of historical data or losing your windowed tallies.
Key Takeaway: Manual savepoints provide a reliable, version-controlled mechanism to pause your streaming state, allowing you to update your application logic or infrastructure while maintaining perfect processing continuity.