I’m looking at using a distributed map to process a JSON file and send chunks of the JSON to a downstream application with Lambda. I have this set up currently and it works great, however, I want to flush out the error logic needed to support this. A basic example of this would be using distributed map to invoke a lambda to push the batches of JSON onto an SQS, same process.
The threshold of errors needs to be very low, <1%, and no duplicate items can be sent during execution.
I’m trying to understand the behavior behind the Map Runs Retriers. I currently have retriers set on the lambda, that would catch ~99% of the issue, but if I set a retrier on the Map Run itself, does it reprocess all items again? Or only the failed executions? I’ve seen documentation around redrive only reprocessing failed execution, but nothing about the retriers.
My goal is to use the catch of the Map Run to send a notification when any executions fully fail, so my threshold will be set very low, but I’m trying to give myself an extra retry of the Map Run + the Lambda retries without having any duplicate items sent