Out of Memory Exception - Unhandled RangeError

447 Views Asked by At

I've created an NodeJS service that reads messages from an ActiveMQ queue, set up remotely, and picks up these messages, processes it and pushes some data to a GUI.

The issue i'm facing is while trying to process these messages on NodeJS end when they come in at a rapid pace to the ActiveMQ queue at approx. 5 messages (JSON) per second and each JSON message is around 18kb in size. The messages that come in are written to a file, saved to an intermediate MSSQL table and once saved, the JSON file is moved to a Processed folder.

The environment setup is:

  • NodeJS version 8.9.4.
  • ActiveMQ version 5.15.4.
  • Java version 1.8.0_171.
  • Three worker processes are created along side the main Node process to handle the message loads.

The workers are created using the node module "Workerpool". https://github.com/josdejong/workerpool

RAM: 2 GB
Processor: Intel Xeon Dual Core processor @2.27GHz 
OS: Windows Server 2008 R2

I keep running into a Unhandled rejection Range Error: Out of Memory Exception, after processing around 3000 messages from ActiveMQ queue, messages pushed to the queue from an external source at frequency of 200ms / message.

The code:


message.readString('utf-8', function (err, body) {
  fs.writeFile('/path/to/writefileto', JSON.stringify(body), function(err) {
  if(err) {
    // handle the error
  } else {
    /* if the file exists in the source path, move to a Processed path */
    if(fs.existsSync('/path/writtento')) {
      fs.rename('/path/writtento', '/path/to/processedDir', function(err) {
         if(!err) {
           console.log("Successful in moving the file to Processed path");
         }
      }
    }
  }

What could be causing this issue?

Please let me know if any other information is required.

0

There are 0 best solutions below