Python concurrent.futures starvation

Question

I'm using Python concurrent.futures, executes parent multi threads and each parent thread execute child threads. When ThreadPoolExecutor is less than number of required parent threads I got starvation and program stuck.

What is the best approach to:

1. Use const ThreadPoolExecutor

2. Do not get into starvation

Please find below example code:

import time
import sys
import concurrent.futures


MAX_THREAD_EXECUTORS = 5
threadPool = concurrent.futures.ThreadPoolExecutor(MAX_THREAD_EXECUTORS)
threads = []
command_threads = []


def main():
    start_tests()
    join_threads()


def start_tests():
    for i in range(1,14):
       threads.append(threadPool.submit(start_test_flow, i))


def start_test_flow(test):
    print(f"Start test flow for: {test}")
    execute_commands()
    join_command_threads()
    

def execute_commands():
    for i in range(1,5):
        command_threads.append(threadPool.submit(start_command, i))


def start_command(command):
    print(f"Start command for: {command}")
    time.sleep(120)


def join_threads():
    for thread in threads:
        result = thread.result()
        print(f"test result={result}")


def join_command_threads():
    for thread in command_threads:
        result = thread.result()
        print(f"command result={result}")

if __name__ == '__main__':
    main()
    sys.exit(0)

Best Regards, Moshe

Answer 1

The minimum number of threads you actually need is non-deterministic and depends on timing, although there is a number (13 + 1, i.e. one thread for each of the parent threads and at least one thread to run a child thread) that will guarantee that you will never stall. What is most likely happening is that you are quickly creating 5 parent threads and then waiting to create further parent threads and child threads because you only have 5 worker threads. But until you are able to create 4 child threads (in execute_commands) and run them to completion, a parent thread will not complete and thus you are stuck.

Now, for example, insert a call to time.sleep(1) in function start_tests as follows:

def start_tests():
    for i in range(1,14):
       threads.append(threadPool.submit(start_test_flow, i))
       time.sleep(1)

This will allow the 4 child threads to be created and there will be some progress. But depending on timing, you may eventually stall. To guarantee that you never stall, you would have to sleep long enough to allow all 4 child threads to complete before attempting to start the next parent thread.

The bottom line is that you just don't have enough worker threads (13 + 1) to guarantee that you won't stall.