Connection terminated unexpectedly while performing multi row insert using pg-promise

43 Views Asked by At

I am using pg-promise for performing a multi row insert of around 800k records into a table in postgres database and facing the following error:

Error: Connection terminated unexpectedly at Connection. (/usr/src/app/node_modules/pg/lib/client.js:132:73) at Object.onceWrapper (node:events:509:28) at Connection.emit (node:events:390:28) at Socket. (/usr/src/app/node_modules/pg/lib/connection.js:63:12) at Socket.emit (node:events:390:28) at TCP. (node:net:687:12)

I have already tried to configure the connection with parameters like idleTimeoutMillis, connectionTimeoutMillis (also tried adding and removing parameters like keepAlive, keepalives_idle, statement_timeout, query_timeout) and still facing the issue.

const db = pgp({
  host: host,
  port: port,
  database: database,
  user: user,
  password: pass,
  idleTimeoutMillis: 0, // tried with multiple values
  connectionTimeoutMillis: 0, // tried with multiple values
  keepAlive: true,
  keepalives_idle: 300,
  statement_timeout: false,
  query_timeout: false,
});

I am able to insert around 300k records without defining any of the extra parameters mentioned above.

Edited:

Following is the postgres logs:

LOG:  server process (PID 680025) was 
terminated by signal 9: Killed
2024-03-31 17:22:51.693 UTC [1] DETAIL:  
Failed process was running: insert into 
table

Looks like the query was killed. I am not sure if this is related to memory as I'am able to bulk insert with less records.

Any help is highly appreciated.

0

There are 0 best solutions below