SQS Message missing

Question

Some Sqs message are missing after pushing back into queue

Hello,

SQS - fifo

I have a sqs and lambda , whenever there is a message in sqs, the lamda will pick it up and process it, while processing if the message meet some business logic condition then the message will be pushed back into sqs, now i have 5 message pushed backed into message , i'm also receiving the response with message_id that the message has been pushed into the queue, the response is something like this

{
  '$metadata': {
    httpStatusCode: 200,
    requestId: '9ed244a2-a557-5e53-a51e-1a9733ad87ed',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  MD5OfMessageBody: 'fa27c5cf69d0bec8f39e8b9a4a427ec9',
  MessageId: '2bc2bfda-850b-479a-a84f-1ecae53f339b',
  SequenceNumber: '18883682857025007616'
}

I get the similar response to all the 5 messages i pushed. But out of those 5 messages only the first 2 messages are available in the sqs, so when lamda triggers again then the lambda only picking up those 2 messages because those 2 are the messages that are available in sqs, my other 3 messages are not available there.

The Message will have same body but with different Deduplication id.

This is the code im using to push the code

export async function pushBackToQueue(modifiedMessage, messageGroupId, messageDeDuplicationId){
    try {
        const sqsClient = new SQSClient();
        const sqsUrl = config.getSQSQueueURL();
        const input = {
            QueueUrl: sqsUrl,   
            MessageBody: JSON.stringify(modifiedMessage), //Object   
            MessageGroupId: messageGroupId,      //A Id which is common among the messages 
            MessageDeduplicationId: messageDeDuplicationId // UUID
        }
        const command = new SendMessageCommand(input);
        const result = await sqsClient.send(command);
        console.log(result)
    } catch (error) {
        SentryObj.captureException(error, {
            level: 'error',
            extra: {
                method: 'pushBackToQueue',
            },
        })
        console.log(error);
    }
    
  }

What im missing here ? why those 3 messages are not available for me? even if i have 20 messages then only first 2 are available others are not available.

Let me know if some more clarification required.

Thanks,

Answer 1

Looks like you've got a bit of a head-scratcher with your SQS FIFO queue. If you're pushing messages back in but only a few are showing up, here are a couple of things to check out:

• Unique Deduplication IDs: Make sure each message has a unique MessageDeduplicationId. SQS can be fussy about this, especially with FIFO queues.
• Visibility Timeout: If your messages are still "invisible" because of the timeout, they won't pop back up immediately. Might be worth checking those settings.
• Check Your Lambda Logic: There might be something in your code that's causing only certain messages to go back into the queue.
• Logs Are Your Friends: Dive into those CloudWatch logs. They can sometimes tell you exactly what's going wrong.
• Queue Settings: Double-check your SQS settings. There might be something there that’s messing with your messages.

Hope this points you in the right direction! Let me know if you need more deets.