My worker in celery automatically gets shut down and logs this message worker: Warm shutdown (MainProcess), and doesn't do anything, I'm running a long running AI/ML tasks in celery
my celery config is something like this:
**app.conf.update(
result_expires=None, # Disable result expiration
worker_max_tasks_per_child=500, # Disable limiting the number of tasks each worker can execute before it's replaced
worker_prefetch_multiplier=0, # Disable prefetching tasks
worker_concurrency=1, # Use all available CPU cores for concurrency
worker_disable_rate_limits=True, # Disable rate limits for tasks
task_track_started=True, # Track started tasks
task_soft_time_limit=None, # Disable soft time limit for tasks
task_time_limit=None, # Disable hard time limit for tasks
worker_max_memory_per_child=None, # Disable memory limit for worker processes
task_acks_late=True,
broker_connection_retry_on_startup=True, # Enable broker connection retries on startup
)**
Sometimes it shuts down and sometimes it doesn't can anyone say something