Coordinating 2 Node JS Cron Jobs and other tasks sequentially

26 Views Asked by At

I have 2 cron jobs in Node JS which monitor 2 distinct resources, The first monitors a queue for products and the other monitors a database of product information so that when data is available for the product the data is retrieved from the queue, the product is enriched and another process generates updates for downstream processes to alert a final system that the product is ready. The problem I am facing is that the steps for finding the ready products, updating and marking them as complete becomes out of sync with the cron jobs that cleanup and update state of the queue. I cannot cause the steps to be in sync with the cron jobs. For example updating state to complete causes the product info to not be available to write the status in the PostGresSql database. Is there a way to synchronize these steps in a way where they processes don't trample each other? I am only looking to successfully coordinate these processes

    const queueReader = new QueueReader()
    
    const sqlUpdateWriter = new PostGresSqlUpdateWriter()

    const streamingProductWriter = new StreamingProductWriter()

    
    const productDescriptorRepository = new ProductDescriptorRepository()
    
    const isReady = await productDescriptorRepository.findById({_id:productId, ready: true}
    
    if(isReady){
      // Do processing of product
      // Find Products that are ready
      // Pull product details from database of ready products
      // Enrich product 
      // Update product status as 'COMPLETE'
    }

    findReadyAndUpdateProducts = async(){
      // find products 
       const productsCompleted = await productDescriptorRepository.findCompletedEnrichments()
      streamingProductWriter.writeWithCursor(productsCompleted)
       
    }
    // Find all {status: 'COMPLETE' and delete them from database
    deleteCompleted = async() => {
       const allCompleted = await productDescriptorRepository.findCompletedById()
       sqlUpdateWriter.deleteAll(allCompleted)
    }

     //Here is the final one of the cron jobs
     const job  = cron.schedule('1-5 * * * *',', findReadyAndUpdateProducts)
     job.start()


    //Here is the one of the cron jobs
     const completedJob  = cron.schedule('1-5 * * * *',', deleteCompleted)
     completedJob.start()
     
   

0

There are 0 best solutions below