How to use Azure Service Bus as Broker and Result for Celery in Django?

Question

I'm currently using Celery with Redis Cloud and it works fine. I want to move to Azure Service Bus, but i'm not finding any docs that describes how to do this. I've seen that celery will use Kombu instead native transport method like AMQP or Redis. Can someone please help me with that?

My main problem is: How to set CELERY_BROKER_URL and CELERY_RESULT_BACKEND considering that i have alredy a Azure Service Bus with a Topic, Subscription and a Queue?

azure-core==1.27.1
azure-servicebus==7.11.0
celery==5.2.7
django-celery-beat==2.5.0
kombu==5.2.4

Answer 1

To use Azure Service Bus as a broker and result for Celery in Django, you have to

• Set the CELERY_BROKER_URL and CELERY_RESULT_BACKEND environment variables.

• Set the CELERY_BROKER_URL to the connection string of your Azure Service Bus queue or topic.

• Set the CELERY_RESULT_BACKEND to the connection string of your Azure Service Bus queue or topic.

Setting the CELERY_BROKER_URL and CELERY_RESULT_BACKEND environment variables in Django settings.py file.

CELERY_BROKER_URL = 'azure-servicebus://<NAMESPACE>.servicebus.windows.net/<QUEUE_OR_TOPIC_NAME>?<QUERY_STRING>'
CELERY_RESULT_BACKEND = 'azure-servicebus://<NAMESPACE>.servicebus.windows.net/<QUEUE_OR_TOPIC_NAME>?<QUERY_STRING>'

Use Azure Service Bus as a broker and CosmosDB or MsSql as a result for Celery in Django. To set up CosmosDB or MsSql as a result backend, you need to install the appropriate Python libraries and configure the connection string in your Django settings.py file.

For CosmosDB, use the azure-cosmos library. To set the CELERY_RESULT_BACKEND environment variable to use CosmosDB.

Check Azure Service Bus client library for Python Code.

CELERY_RESULT_BACKEND = 'azure-cosmos://<COSMOSDB_ACCOUNT_NAME>:<COSMOSDB_ACCOUNT_KEY>@<COSMOSDB_ACCOUNT_NAME>.documents.azure.com:443/<DATABASE_NAME>?ssl=true'

For MsSql, use the pyodbc library. To set the CELERY_RESULT_BACKEND environment variable to use MsSql.

CELERY_RESULT_BACKEND = 'mssql+pyodbc://<USERNAME>:<PASSWORD>@<SERVER_NAME>/<DATABASE_NAME>?driver=ODBC+Driver+17+for+SQL+Server'

For more information, refer to Azure Service Bus -Python and SO Link.