How to handle ECONNRESET when proxied response includes `connection: close` header?

Question

I am attempting to implement a MITM proxy.

I am handing a CONNECT request which is then used to create a connection with an internal HTTPS server.

Upon request, HTTPS server responds with:

connection: close
foo

I expect the client to receive the response and proxy to close the connection socket.

Instead, client receives the response and proxy server logs an error:

server socket error Error: This socket has been ended by the other party
    at Socket.writeAfterFIN [as write] (net.js:407:14)
    at Socket.ondata (_stream_readable.js:713:22)
    at Socket.emit (events.js:200:13)
    at addChunk (_stream_readable.js:294:12)
    at readableAddChunk (_stream_readable.js:275:11)
    at Socket.Readable.push (_stream_readable.js:210:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:166:17) {
  code: 'EPIPE'
}
request socket error Error: read ECONNRESET
    at TCP.onStreamRead (internal/stream_base_commons.js:183:27) {
  errno: 'ECONNRESET',
  code: 'ECONNRESET',
  syscall: 'read'
}

Here is the subject script:

const net = require('net');
const http = require('http');
const https = require('https');

const sslCertificate = {
  ca: '-----BEGIN CERTIFICATE REQUEST-----\n' +
    'MIICWTCCAUECAQAwFDESMBAGA1UEAwwJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0B\n' +
    'AQEFAAOCAQ8AMIIBCgKCAQEAuftLzDyJ8dRk71pZ3637tCIZCVLJieLqIlAf7wT5\n' +
    '+qesTgu6vWzndZ4ze2V2lkac0xqFlW1djKT9IPUTCPx5dmWdT8mYFNUqB87hRWx9\n' +
    '6Ge21bs+KDppujHYrrgNjT8L3+RlHenoG7Qi5WuSzfOqP5nqCyoKFFNHJ0Ds52Uk\n' +
    'uvmTLzY/+kx3tFFGi4QXyva3T38uF99D4C2Tqxy7aRHEBJATQYxJgVPResiv31zv\n' +
    'qd6H1jYIZGw5s4QJFh5C7VXsoHs1dLIfDoNcV/fO95VQ+wXPxrl8mcVQzNV7RKmX\n' +
    'VHKudzx49IvOpRyM3OmN3RV5snOYKGmgwXQUF7JL2VSrSQIDAQABoAAwDQYJKoZI\n' +
    'hvcNAQELBQADggEBAIaUryumwXIxMJErT/7B46l2k27+xefaTPCddjERhqk8WH/N\n' +
    '95/yhvdzq1i0BSLv74Kh7L68kJiN8vtF6sAORofw42LMo+KzRDE1m1Zl7CVWw2DF\n' +
    'wT7SJov22t6dVx6HOcsZZSo5lSN+CMN3xkgt6jyEPbCKfCJzl44Y3eOpqzry6/GM\n' +
    'U+hR7nQx3IJmpAHNd7wolRzkf1X0gTifR5iC5S72GSRM9AnLfL2L0zQC6LmcNmZp\n' +
    '3deNxIC+w5kTALREiMq3P9McBMCgwRinOJLbhmV9ifPRpLa9e+mFVdHzbR7+09kp\n' +
    '6eNS19RndbHn6N1RbgFSNjDz28fMXISSWZFB/X4=\n-----END CERTIFICATE ' +
    'REQUEST-----',
  cert: '-----BEGIN CERTIFICATE-----\n' +
    'MIICpDCCAYwCCQCK9kDE6/eFXDANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDDAls\n' +
    'b2NhbGhvc3QwHhcNMTkwNzE5MTczMzI2WhcNMjAwNzE4MTczMzI2WjAUMRIwEAYD\n' +
    'VQQDDAlsb2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC5\n' +
    '+0vMPInx1GTvWlnfrfu0IhkJUsmJ4uoiUB/vBPn6p6xOC7q9bOd1njN7ZXaWRpzT\n' +
    'GoWVbV2MpP0g9RMI/Hl2ZZ1PyZgU1SoHzuFFbH3oZ7bVuz4oOmm6MdiuuA2NPwvf\n' +
    '5GUd6egbtCLla5LN86o/meoLKgoUU0cnQOznZSS6+ZMvNj/6THe0UUaLhBfK9rdP\n' +
    'fy4X30PgLZOrHLtpEcQEkBNBjEmBU9F6yK/fXO+p3ofWNghkbDmzhAkWHkLtVeyg\n' +
    'ezV0sh8Og1xX9873lVD7Bc/GuXyZxVDM1XtEqZdUcq53PHj0i86lHIzc6Y3dFXmy\n' +
    'c5goaaDBdBQXskvZVKtJAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAGxXxytrNtm+\n' +
    'q4NpWtKhy3DL5LOMH+K8lqgJ29SmmDEcqWgevpUnqLYFvb3AOxU/vYId5rFmHb5A\n' +
    'WnXyKJ/YYSpNi47EcV+AJCwqDqBgAM4J3Tiiu6BguZ4sU20ZVFl1oQvTlQw8InLI\n' +
    'D1ciwwtgWS2z9pRKmQ2ar2TY+2yhnl0L1WCl50XH6PngzzEHSxHiPDnOYPyXQjPs\n' +
    'vkoJDmdnAVfWs2DfKfM0l27nIL2IBZr6Gks+nLwaK7FedQVD8ORYg9x/mwXO1oDr\n' +
    'sLyCQUlXhhBNBmn+TTLFPbrXetOU6le7iW3JJVMUv84vh8cV8aLtXDuQ0qlKMd8B\n' +
    'Mrgha3mM8EM=\n-----END CERTIFICATE-----',
  key: '-----BEGIN RSA PRIVATE KEY-----\n' +
    'MIIEpAIBAAKCAQEAuftLzDyJ8dRk71pZ3637tCIZCVLJieLqIlAf7wT5+qesTgu6\n' +
    'vWzndZ4ze2V2lkac0xqFlW1djKT9IPUTCPx5dmWdT8mYFNUqB87hRWx96Ge21bs+\n' +
    'KDppujHYrrgNjT8L3+RlHenoG7Qi5WuSzfOqP5nqCyoKFFNHJ0Ds52UkuvmTLzY/\n' +
    '+kx3tFFGi4QXyva3T38uF99D4C2Tqxy7aRHEBJATQYxJgVPResiv31zvqd6H1jYI\n' +
    'ZGw5s4QJFh5C7VXsoHs1dLIfDoNcV/fO95VQ+wXPxrl8mcVQzNV7RKmXVHKudzx4\n' +
    '9IvOpRyM3OmN3RV5snOYKGmgwXQUF7JL2VSrSQIDAQABAoIBAGnWDuFwBhQ/iR0I\n' +
    'rqJy0Q1GZjb/DL/SCOlz7WhIzbUNnClh1WgcxG8TkzqCmASWtIIR0rkhXp49+eq6\n' +
    'bJWtj7WHyAjysQAR+nQtD9dBETmjY9GnV4zvCOGzohpzlQqvOSO1RrHKPZMeZMln\n' +
    '+UgIhPbisOSfjNLaPWCiOu7HiSp5CgT70mSrylNQWhIa/okt8zjDbpV4QGPYP8J/\n' +
    'fi4k3u5C8oHwCt3DYp4Qc6ybKiMuBELVcoI0Ug0CtVriB11uNCYOqMbanj4VfRzq\n' +
    'KPTDRtkiF+EYi0PBstW+X9p7rFVB1PaBSF3PxudWMTmNZ1MooqOfkIves/T7YoxA\n' +
    'Uh9XUIECgYEA4Y8RU+/lf5GMDKstdwm+OH0NBOT/mrsFAlWnGtQivWddyPxvPVJH\n' +
    'LqIYtpqTH2luh7ksTcmTacqRjFx/ebobFAVgvg1zhCzHIdmgedGuxzvhEic1KRgT\n' +
    'EJgm4kW9uPFZugd05873uWf0cYjbQZXQhhn1E3bTorTuJJZoJu0R7ZECgYEA0xTb\n' +
    'bnFyOgD+c0A+kkirHiYU5RDAvtCS0jyKbZAPTP3fX016JeC2pxQcN4iLvgumm+Iv\n' +
    'ugtdrHYDzZTIzMl0pT8HSDqjaW8nNmEMvYaE8FYGlFHqEJQlweGMYeXdCxZSA+1D\n' +
    'HAzG8tW0rniMZp6KevZt5GCmBX3q0mH9ZKU/ZjkCgYEA14JTgwhOFXHiBuSyvu6v\n' +
    'MdfBTbDiy1rvMUjXLZoMSz1s7TDLtCJd4p97z1SnRzb8JW92dign0cd7A0oJfiuj\n' +
    '3aA5y7ycZ2hFJwGBA4OlY7TBmg+eClJ3PL6zQDR0TjVDjqu7NhSYuiwp8SRaoTJc\n' +
    'FxTMBTnegbIvawPOJYsTOxECgYEAlVDzyLTHsPBzDuQrXx+4rKMTtNadAl5Y/g+F\n' +
    'fOujZztPgAM2nQTRMG+xZjdZYx6qxSrDyD+yDAWPuyW8xeDceuiTJi0U28idXIJa\n' +
    'mNdHwxuXm+Q2R3QFIZmDzNzl+KnZap20E2uWcMFsBt+PsigEneck5aDY0Jm6OwjG\n' +
    'TyP2LUECgYALK+5AoQYbeUwVd3MhJONl0EdtKzjDq2wI127oXCjqVIe9BoqNedDu\n' +
    'zOvo5QjNApRbPZcaJB7e/3XbMFv/jSpeL9jC/AynGQBdpk3meL9KtC7Nm4wwj8XX\n' +
    'Ad5ZZkUZLAukbH1BqBuEgFjv3SDJ2g/aqUdqVfwq6qNSNdWzZTQG4w==\n-----END RSA ' +
    'PRIVATE KEY-----'
};

const handleConnect = (port, request, requestSocket, head) => {
  const {
    httpVersion
  } = request;

  const serverSocket = net.connect({
    port
  }, () => {
    requestSocket.write(
      'HTTP/' + httpVersion + ' 200 Connection established\r\n' +
      '\r\n'
    );

    serverSocket.write(head);

    serverSocket.pipe(requestSocket);
    requestSocket.pipe(serverSocket);
  });

  serverSocket.on('error', (error) => {
    console.log('server socket error', error);
  });

  requestSocket.on('error', (error) => {
    console.log('request socket error', error);
  });
};

const requestHandler = (incomingMessage, outgoingMessage) => {
  outgoingMessage.writeHead(200, {
    connection: 'close'
  });
  outgoingMessage.end(Buffer.from('foo'));
};

const main = () => {
  const httpServer = http.createServer(requestHandler);

  const internalHttpsServer = https
    .createServer(sslCertificate, requestHandler)
    .listen()
    .unref();

  httpServer.on('connect', (request, requestSocket, head) => {
    handleConnect(
      internalHttpsServer.address().port,
      request,
      requestSocket,
      head
    );
  });

  httpServer.listen(8080);
};

main();

This script can be tested with any HTTPS URL, e.g.

curl --proxy http://127.0.0.1:8080 'https://127.0.0.1/' -k

Alternatively, you can:

git clone https://github.com/gajus/http-proxy-connection-close.git
cd ./http-proxy-connection-close
node ./server.js
curl --proxy http://127.0.0.1:8080 'https://127.0.0.1/' -k

How to handle ECONNRESET error when proxied response includes connection: close header?

Answer 1

There were two unrelated issues.

The ECONNRESET can be prevented by explicitly destroying the client socket on "end" event.

serverSocket.once('end', () => {
  clientSocket.destroy();
});

However, upon stress testing this implementation I started to see ECONNREFUSED errors.

Turns out that sometimes the IPC connection would fail:

errno:   ECONNREFUSED
code:    ECONNREFUSED
syscall: connect
address: /tmp/raygun-cjyaj51hz000035h3erhlf7a3.sock
name:    Error
message: connect ECONNREFUSED /tmp/raygun-cjyaj51hz000035h3erhlf7a3.sock
stack:
  """
    Error: connect ECONNREFUSED /tmp/raygun-cjyaj51hz000035h3erhlf7a3.sock
        at PipeConnectWrap.afterConnect [as oncomplete] (net.js:1054:14)
  """

This would happen predominantly when CPU usage is near 100%.

In those instances, the client socket would just hang waiting for connection with the internal HTTPS server.

The solution is one of the two:

1. Terminate the socket
2. Retry the connection with the internal HTTPS server

This is what the former would look like:

serverSocket.on('error', (error) => {
  log.error({
    error: serializeError(error)
  }, 'server socket error');

  clientSocket.write([
    'HTTP/1.1 503 Service Unavailable',
    'connection: close'
  ].join('\n') + '\n\n');

  clientSocket.end();
});

https://gist.github.com/gajus/72270a9f3aea3b09d61b997f7e5537f3