I am facing an issue related to job bookmark(enable/disable/pause) in step function.
Scenario
I have a glue job name "myjob1" with having configuration enabled job bookmark by manually/selecting from drop down in job details.
Now my case is I want to run single glue job multiple times by passing different parameters via step function.
Here I am able to run parallelly multiple tables but in this case job bookmark is not working properly and creating duplicates at s3 output.
Approaches Tried
1.Bookmark working successfully in glue job when passed from drop down in job detail section. 2.Bookmark is not working when passed from parameter(--job-bookmark-option:job-bookmark-enable). 3.Bookmark is not working when one glue job is run from step function and bookmark is enabled from drop down in job detail section.
Any suggestion/solution will be highly appreciated. Thanks in advance!!!
Bookmark functionality (enable/disable/pause) with step function should work for my concurrent running same glue job.