Can you pass multiple arguments into jobs.put?

Question

So I want to use the queue multithreading in Python, where I queue jobs and it works through the jobs but I have two values I would like to pass to each job. Can I pass multiple values into the job queue, and how do you retrieve the data?

I tried the following:

def do_stuff(q, t):
    while not q.empty():
        value = q.get(0)
        orgName = q.get(1)
        slpTmr = random.randint(1, 3)
        time.sleep(slpTmr)
        print(str(value) + "[" + orgName + "] slept for: " + str(slpTmr))
        q.task_done()

for org in orgs_search: 
    org_id = org["id"]
    org_name = org["name"]

    if org_id == 0:
        continue

    jobs.put(org_id, org_name)

for i in range(maxThreads):
    worker = threading.Thread(target=do_stuff, args=(jobs))
    worker.start()

but it doesn't print as expected, what I would hope for is an output of this:

123 [org1] slept for 2 
142 [org2] slept for 4 
123 [org3] slept for 3 
342 [org4] slept for 1 

Answer 1

You should be using the ThreadPoolExecutor() or multiprocessing ThreadPool library objects instead of rolling your own, but for the sake of exercise:

• just pass in the queue to your workers
• pass in the work item as a tuple (don't .get() twice per item!)
• be sure to join() the queue and the worker threads.

import random
import threading
import time
from queue import Queue


def do_stuff(queue):
    while not queue.empty():
        value, orgName = queue.get()
        slpTmr = random.uniform(1, 3)
        time.sleep(slpTmr)
        print(f"{value} [{orgName}] slept for {slpTmr}")
        queue.task_done()


def main():
    jobs = Queue()
    orgs_search = [
        {"id": 0, "name": "Apple"},
        {"id": 1, "name": "Microsoft"},
        {"id": 2, "name": "Google"},
        {"id": 3, "name": "Disney"},
    ]
    for org in orgs_search:
        org_id = org["id"]
        org_name = org["name"]

        if org_id == 0:
            continue

        jobs.put((org_id, org_name))

    max_threads = 2
    workers = [threading.Thread(target=do_stuff, args=(jobs,)) for _ in range(max_threads)]
    for worker in workers:
        worker.start()
    jobs.join()
    print("Jobs are done!")
    for worker in workers:
        worker.join()
    print("Workers are done!")


main()