import multiprocessing
from time import time
def do_something(arg):
# do something here
def main():
t1 = time()
pool = multiprocessing.Pool(processes=2)
t2 = time()
# Launch async processes to do something
futures = pool.starmap_async(do_something, [a, b])
# do some other task
# Wait for processes to join
pool.close()
pool.join()
if __name__ == "__main__":
for i in range(10):
main()
The function main() is run multiple times to do some task and spawns 2 processes to do some other task in parallel, while the main() function performs its own task. However, over different runs the time (t2 - t1) is sometimes in the order of ~20 ms and other times it is approximately ~3-4s. I want to understand any possible reasons this could be happening.
I have gone through the documentation, but couldn't find a good reason. My system is a 12-core system and I have made sure that the system occupancy is low (I have tried running this piece of code after a reboot). Any help is highly appreciated.