Health Check API blocked due to single IOLoop

Question

I have a tornado application, we have two API's /health and /make call to /make API takes 10 min to build required resources and load it to memory, during which period call's to /health is blocked due to which the server is marked as unhealthy. what is the better way to build /health API.

Answer 1

It's a good and widespread practice (for a reason) to move long blocking operations from the main thread into a separate thread/pools/Celery/etc. If you do so with resources building, your main thread with the /health would be unblocked and available.

Answer 2

I believe that the easiest, and most tornado-like, way of moving the blocking process into a new thread is to use tornados subprocess impl. which described here: https://www.tornadoweb.org/en/stable/process.html#tornado.process.Subprocess

In short: the idea is to start the build process in a new thread where the I/O is added to the IOLoop like any other non-blocking I/O resource. In reality the new process (the child / sub process) is completely separate from the main tornado process but it's interfaced to hide that fact.