Nodejs Spawn not working with js file that works manually

Question

Ok so i have a main application that runs an express server from this express server i have an end point that accepts a get request with an id

when the user requests this end point a function is run to spawn a detached process that accepts the id as a arg and then starts downloading a list of files in the background

now my downloader js file works perfect no errors at all if i run it manually from the terminal passing the args there using

node downloader argid

it outputs all the correct stuff and everything is downloaded

but when i try and run this same downloader file from within my main script with this code

    downloaderSpawn() {
      const nodeBinary = '/usr/bin/node';
      const childScriptPath = '/home/ebo/downloader.js';
      const logFilePath = `/home/ebo/${this.id}/logfile.txt`;

      const command = `${nodeBinary} ${childScriptPath} ${this.id} ${this.key} > ${logFilePath} 2>&1`;

      const childProcess = exec(command);

      childProcess.stdout.on('data', (data) => {
        console.log(`Child Script Output: ${data}`);
      });

      childProcess.stderr.on('data', (data) => {
        console.error(`Child Script Error: ${data}`);
      });

      childProcess.on('close', (code) => {
        console.log(`Child process exited with code ${code}`);
      });
    }

i have tried with and with out the > ${logFilePath} 2>&1 option the log file is created but never populated with the output of downloader telling me that the script is never run

this function is from inside a larger class everything else in this class works as intended apart from this function and as stated the downloader js file works fine when called on its own

i am forced to currently use ubuntu 20 as this is what the client has provided the version of node in the ubuntu 20 repo seems to be 18.19.0

i have tried using exec and spawn i have also tried detaching the process using

const childProcess = spawn(command, {
  shell: true,
  detached: true,
  stdio: 'ignore', // Optionally ignore stdio if not needed
});

i get 0 errors or output of any kind in the main script console or in the logfile its been set to redirect to

EDIT : I have upgraded to node version 21 still the same issue the child script does not seem to run at all from spawn / exec

Answer 1

Just use fork and do that correctly:

downloaderSpawn() {

  // you don't need this: fork executes node instance automatically
  // const nodeBinary = '/usr/bin/node';

  const childScriptPath = '/home/ebo/downloader.js';
  const logFilePath = `/home/ebo/${this.id}/logfile.txt`;
  
  // control logging from code, instead delegate that to the bash
  const logStream = fs.createWriteStream('logFilePath');

  // you don't need this 
  // const command = `${nodeBinary} ${childScriptPath}...
  
  // pass arguments as array to fork
  const childProcess = fork(childScriptPath, [this.id, this.key], {
    // if you need to exchange between processes data types, 
    // like BigInt, Typed Arrays, Map, Set, etc
    serialization: 'advanced',
    // needed for get access to child process stdout and stderr
    stdio: 'pipe'
  });

  // use 'message' event and send method for exchange typed data
  // instead just a strings
  childProcess.on('message', msg => {
    console.log(
      `msg, sended to parent, using process.send(msg): ${msg}`
    );
  })

  childProcess.stdout.on('data', (data) => {
    console.log(
      `Child Script Output using console.log, or process.stdout.write: ${data}`
    );
    logStream.write(data.toString('utf8'));
  });

  childProcess.stderr.on('data', (data) => {
    console.error(
      `Child Script Output, using console.error, or process.stderr.write: ${data}`
    );
    logStream.write(data.toString('utf8'));
  });

  childProcess.on('close', (code) => {
    console.log(`Child process exited with code ${code}`);
    logStream.close();
  });
}