how to initialize a different object for different sub-process of tornado-based application?

Question

The following are my code:

class PromptHandler(tornado.web.RequestHandler):
    def initialize(self, obj):
        self.obj = obj
    async def get(self):
        self.write('OK')

def make_app(obj):
    return tornado.web.Application([(r'/index', PromptHandler, {"obj": obj})])

async def post_fork_main(obj):
    app = make_app(obj)
    server = tornado.httpserver.HTTPServer(app)
    server.add_sockets(sockets)
    await asyncio.Event().wait()


class MyClass:
    def __init__(self):
        pass


if __name__ == '__main__':
    sockets = tornado.netutil.bind_sockets(8080)
    tornado.process.fork_processes(4)
    obj = MyClass()
    print(f"process id is {os.getpid()}, the obj id is {id(obj)}")
    asyncio.run(post_fork_main(obj))

Then I get the following result:

process id is 17035, the obj id is 4373544288
process id is 17036, the obj id is 4373544288
process id is 17037, the obj id is 4373544288
process id is 17038, the obj id is 4373544288

There are 4 sub-processes. I hope I can initialize a different instance for every sub-process. But the IDs for the instance are the same. It seems they are the same instance. How do initialize a different instance for every sub-process? Thanks.

Answer 1

The result you see is exactly what I would expect, you instantiate obj and then pass it to each subprocess.

What happens behind the scenes is that obj gets pickled and is passed to each subprocess and then un-pickled back into a python object. With obj.id() returning the same value because logically it's the same object.

If you instantiated object or made a copy of it inside post_fork_main(...), then you would see a different identifier.

You question leads me to ask one in return. Why are you forking Tornado in the first place? Are you wanting to split work across CPUs? There are other approaches probably better suited to distributing incoming requests to tornado and queuing them to out of process workers.