I have a project in which I run multiple processes to do multiple jobs, the jobs are distributed via a queue.
queue = multiprocessing.Queue()
process = multiprocessing.spawn(
run_alg,
args=(queue,),
nprocs=process_num,
join=False,
)
process.join()
However, it is possible for the processes to stop working or to crash mid-operation. I want to create a pool of x processes where every time a process is shut down, a new process will be created to take its place until the queue is empty.
Is it possible, how?