NServiceBus: Subscribe to AWS Textract event via SQS

110 Views Asked by At

As the subject states: I'm using Amazon Textract to detect text on scanned documents. When Textract is done, it fires of a notification via SNS and the SQS queue subscribes to the notification.

I can reveive the "all done" event with the AmazonSQSClient but since the rest of the solution is built on top of NServiceBus I want to handle the native AWS events via NServiceBus as well.

How do I tell the SqsTransport to subscribe to a single, already existing SQS queue? I don't want nsb to create new queues and I don't intend to send messages via this queue. This endpoint is meant to be "receive only".


UPDATE

My problem is not "how do I get a native AWS event off the wire" but "how do I configure NSB to use my existing queue instead of creating its own".

UPDATE2 Turns out my problems is not only how to configure nsb. Getting the native AWS event off the wire is another obstacle that the documentation on native integration does not completely solve.

3

There are 3 best solutions below

1
Sean Farmar On

Will this help? You will have to do the native to NSB consuming: https://docs.particular.net/transports/sqs/native-integration

2
Daniel Marbach On

By default, NServiceBus translates the endpoint name into the queue name. So if your endpoint name is, for example, Sales.OrderProcessing this will be translated to Sales-OrderProcessing by the Amazon SQS transport to make it compatible with the transport specific "restrictions".

It is possible to decouple the endpoint name from the queue name by overriding the local address of the endpoint

endpointConfiguration.OverrideLocalAddress("someothername");

https://docs.particular.net/nservicebus/endpoints/specify-endpoint-name

1
Sebastian Weber On

While @DanielMarbach's answer helped it did not solve all the problems. I had to set a custom QueueNameGenerator on the transport to fit my existing queues.

The validation of the timeout settings for the delayed message queue (which I honestly neither need nor want in my specific case) were rather hard to figure out because the validation inside NSB does not really fit the error message. Delay is 00:01:00 but must be less than 00:15:00... eh, what?! I had to set the timeout to the maximum of 15 minutes that SQS allows me.

The harder obstacle was that the SqsTransport (specifically the InputQueuePump) requires the MessageTypeFullName as part of the MessageAttributes of the native AWS message (or at least I could not find another way through the maze of message intake that worked with the notifications I get from Textract). I tried a custom Behavior but could not find a stage early enough in the pipeline to work for me. I tried to customize the deserialization into TransportMessage but the message pump uses an internal System.Text.Json.JsonSerializer instance that does not care about settings or customizations of the global message serialization. What I ended up doing was to write a decorator for the IAmazonSqs client the transport uses and intercept the ReceiveMessageAsync method. Since I only plan to use SQS for these notifications it was as simple as adding the AQTN of the message as a MessageAttribute of the received message. Long story short: It is a much more involved process than I hoped but now I can hook up to the Textract notifications via NSB.