How to start cancelled Apache Flink Job from the old external checkpoint

Question

I am running an apache Flink stream job which consume events from one kafka cluster and produces to another kafka cluster. And I have externalised the checkpoints so that the state can be recovered from the persisted checkpoints, however when I cancel and start the job it is starting from the new checkpoint instead of restoring from the old existing checkpoint. Is there any way we can restore new jobId from old checkpoint.

I was canceling the new flink job and expecting it to restore the state from an old checkpoint.

Answer 1

Is there any way we can restore new jobId from old checkpoint?

Yes. When you start the new job, specify which checkpoint you want it start from using the
--fromSavepoint option, pointing to the old checkpoint. Although this option sounds like it might only work with savepoints, it also works with retained (externalized) checkpoints.

See the documentation for all the details. By the way, this option is also available from the REST API.