Fair Queues with ActiveMQ Artemis?

Question

We are trying to create fair queues with ActiveMQ Artemis.

Say company A sends a lot of msgs to process. Now company B sends msgs. We want msgs from company B to be processed before all of the previously enqueued msgs of company A are done. In a fair, parallel manner. So company B doesn't have to wait for A.

It would be nice to not make separate queues for A and B. Just one for a certain consumer process.

We have used 'Queue-Selector's by setting-up consumers for each company and adding the company prop to the msg. It has been observed to worked really fairly but we are seeing under load the second company's msgs are delayed. We suspect the journal files get large enough and the broker can't see them all to send the consumer with the selector the backed up msg.

So wondering if anyone has solved that with a feature of ActiveMQ Artemis.

Thanks!

Queue-Selector not working.

Answer 1

The essential semantics of a queue are first-in-first-out (i.e. FIFO). This is typically considered fair since the first message to arrive in the queue is the first to leave. You can, of course, override this a bit with different message priorities, but there is no support for the specific behavior you're describing and I'm not sure if there was that it would be called fair.

Even with consumer-based selectors the queue still adheres to FIFO semantics so it's not surprising that you still see delays.

The simplest solution (which you want to avoid for some reason) is just to use different queues. You could leverage the power of multicast, filtered queues, and fully-qualified-queue-names to make this as simple as possible. For example, you could have a single address where all messages were sent and multiple queues with filters which would collect the messages according to a property value unique to the sending company. Corresponding configuration for companies A and B using a property named company would look something like this broker.xml:

<addresses>
   <address name="allCompanies">
      <multicast>
         <queue name="A">
            <filter string="company='A'"/>
         </queue>
         <queue name="B">
            <filter string="company='B'"/>
         </queue>
      </multicast>
   </address>
</addresses>

So all companies would send messages to the allCompanies address. However, consumers who wanted the messages from one particular company would receive from the company's specific queue (i.e. allCompanies::A or allCompanies::B).