How to use multiple queue worker with Shopware 6?

Question

Are there any best practive in running multiple message queue workers in Shopware 6? We disabled the Admin Worker und configured supervisor.d to run 5 processes of messenger:consume and 1 process of scheduled-task:run . We experienced some weird issue: Enqueue seems to have some 20 minute processing time limit (redeliver_after). After 20 minutes of processing a job seems to be considered stale and any other worker is free to take responsibility for it. We also had infinite loops of message dispatching. The whole collaboration between enqueue, symfony/messenger and Shopware is so complex and Shopware documentation is so parse on that.

Answer 1

Due to nobody else posting an answer, I'll answer this one myself. What we discovered: Even with just one worker jobs running more than 20 minutes cause constraint violations on table message_queue_stats. We have one or two jobs which exceeds this limit. We will try to increase this limit. Futhermore we encounter deadlocks in MySQL on enqueue table. I guess that needs further investigation. Finally we switched to Redis for queue persistence. You have two options here: Redis Adapter from Symfony Messenger or Redis Adapter for Enqueue. We tried both and it worked better with Enqueues Redis Adapter. Keep in mind Shopware advises to use RabbitMQ for having multiple workers.