We have dockerized Rails application and It is deployed on AWS(using Fargate).
Currently, we have Dev and Staging servers running on AWS. We use Resque for scheduling and queuing purpose. We do use foreman to start below processes once deployment is completed.
#Docker CMD bundle exec foreman start -f Procfile.dev-server
#Procfile.dev-server web: bundle exec rails s -p 3000 -b '0.0.0.0' queues: bin/rake environment resque:work QUEUE=responsive_tasks scheduler: bin/rake environment resque:scheduler
This setup is good and working fine till staging environment. Foreman is not recommended for Production setup and might be issue when scaling would require as all 3 processes running on same machine.
Doubts
How I can run all 3 different processes(web server, queues and schedulers) without Foreman once code is deployed on AWS for production?
Shall I run Resque servers on separate instance with same Rails app code?
You can create separate ECS services for each of your processes i.e. web-server, queue, scheduler and leverage ECS Service Discovery feature for service-to-service communication. By doing this, you can manage the lifecycle of these individual processes separately and also perform application-autoscaling on each of these services separately.
Note: Initiate your processes within the Docker ENTRYPOINT to make sure that the processes start during the container creation. By doing this, you will no longer require a process manager daemon like Foreman to manage your processes. The lifecycle of the processes (tied to container lifecycle) will be managed by the ECS service.
Please add a comment if you need further clarification on the above approach.