Spring asynchronous @Retryable not working SOMETIMES when deployed on multiple AWS EC2 instances

83 Views Asked by At

As title mentioning, I am having a CompletableFuture @Async and @Retryable annotated method, it works totally fine at my local but when source deployed on EC2 instances, sometimes it won't retry when have exception.

My source code:

    @Async
    @Retryable(value = {Exception.class}, maxAttempts = 10, backoff = @Backoff(delay = 5000))
    @Transactional
    public CompletableFuture<Item> createItemAsync(ItemCode itemCode, Info info) {
        
        // This line causes Not Found exception frequently since need to wait other Service done.
        Item item = ItemRepository.findBy(itemCode);
        
        // Do some logic...

        return CompletableFuture.completedFuture(item);
    }

When it deployed on AWS EC2 instance, sometimes it will stop retry right after exception thrown on first time. I am not sure why it stop retrying very early and even callback function whenComplete of CompletableFuture wasn't invoked when retry stopped unexpectedly. At other class that invoke async method:

    ItemImageService.createItemAsync(itemCode, info)
                .whenComplete((item,es) -> {
                    if(Objects.isNull(item)) {
                        logger.error(String.format("Error creating item image", item.asText()));
                        return;
                    }
                    itemCompletionEventPublisher.publish(item);
                    }
                });

Is there any possible situations or solution that can prevent this kind of issue? Thank you for helping.

0

There are 0 best solutions below