Opening Scenario
A Flink job has been running uninterrupted for 4 months. The team needs to deploy a new version of the job that adds a new operator and changes the parallelism of an existing one. The job holds 300 GB of keyed state. Stopping the job, throwing the state away, and starting fresh isn't acceptable — that would re-process months of Kafka history (or worse, lose state that came from now-deleted upstream sources). The team also wants to test the new version against a copy of production data without disturbing the running job. Both problems point to savepoints — but the team is unclear when savepoints are the right tool versus relying on checkpoints, and what can and cannot be changed safely between a savepoint and a restore.
Question Progression
1. What is a savepoint, and how is it different from a checkpoint?
Question. Both savepoints and checkpoints capture the full state of a Flink job. They use the same snapshot mechanism. Why does…
In the full tutorial
- When to Reach for This