Spawning NodeJS worker thread inside of async function doesn't respect promise resolution

Question

I've got an async function that launches a NodeJS worker thread like so:

    encode : async (config) => {

      if (isMainThread) {

            const encode_worker = new Worker(`./service-encode.js`, { workerData: config });

            encode_worker.on('message', (transcode_data) => { 
              log.info("%o", transcode_data);
              return transcode_data;
            });

            encode_worker.on('error', (err) => { log.error(err)});
            encode_worker.on('exit', (code) => {
              if (code !== 0)
                throw new Error(`Encoding stopped with exit code [ ${code} ]`);
                console.log("* * * EXITED ENCODER WORKER * * *")

            });

      }

    },

In the serivce-encode.js file I've got the following code which uses async functions. Note that I am using postMessage to signal that it is done.

var transcoder        = require('./transcoder');

const {Worker, isMainThread, parentPort, workerData} = require('worker_threads');

console.log("* * * STARTING ENCODE THREAD * * *\n");
console.log(workerData);

transcoder.run(workerData)
          .then((results) => {
              transcode_data = results;
              parentPort.postMessage(transcode_data);
          })
          .catch(err => { throw err });

Then, I use the following example code but the code in the 'message' event from above fires off immediately. That is, it doesn't seem to wait until it's done:

encode(conf).then((encode_data) => { console.log("Encode Data :", encode_data); 

The encode function works fine, but the console.log statement executes immediately when calling encode() function — also the encode_data var is undefined. Since the return statement in the encode is in the message event, shouldn't the promise of the async function be resolved at that time?

Answer 1

So, NOTHING about the code inside your async function supports promises. You can't just throw random asynchronous (but not promise-based) code inside an async function and expect anything to work. An async function will work just fine with promise-based asynchronous functions that you await. Otherwise, it knows nothing about your asynchronous operations in there. That's why calling encode() returns immediately without waiting for anything to complete.

In addition, return transcode_data is inside an asynchronous callback. Returning inside that callback just goes back into the system code that called the callback and is dutifully ignored. You're not returning anything there from the async function itself. You're returning to that callback.

Since your operation is not promise-based, to solve this, you will have to make it promise-based by wrapping it in a promise and then manually resolved or rejecting that promise when needed with the proper values. You can do that like this:

encode: (config) => {

    if (isMainThread) {

        return new Promise((resolve, reject) => {
            const encode_worker = new Worker(`./service-encode.js`, { workerData: config });

            encode_worker.on('message', (transcode_data) => {
                log.info("%o", transcode_data);
                resolve(transcode_data);
            });

            encode_worker.on('error', (err) => {
                log.error(err)
                reject(err);
            });

            encode_worker.on('exit', (code) => {
                if (code !== 0) {
                    reject(new Error(`Encoding stopped with exit code [ ${code} ]`));
                }
                console.log("* * * EXITED ENCODER WORKER * * *")
            });

        });


    } else {
        // should return a promise from all calling paths
        return Promise.reject(new Error("Can only call encode() from main thread"));
    }

},

FYI, this code assumes that the "result" you're looking for here from the promise is the transcode_data you get with the first message from this worker.