Unhandled critical extension

Question

We are trying to create a node.js application which should interact with a server over HTTPS (>TLS v1.2). We are given some list of key, cert files to establish a connection with the server. Node HTTPS requires CA, cert, key files which are CA file, client certificate, and key files. When provided these we are getting the following error:

Error: unhandled critical extension.

After spending some time on the internet, we found that the Server cert that was received at the time of TLS handshake has some custom extensions. Later when we did openssl verify -CAfile ca_file.pem client_cert.pem we could reproduce it:

error 34 at 0 depth lookup:unhandled critical extension
OK

So, this seems something to do with OpenSSL. How do we make OpenSSL understand our custom extensions? Those custom extensions are critical as well, so that we cannot just ignore the error by setting -ignore_critical.

Edit:

CA certificate has the following in its X509v3 extensions section,

X509v3 extensions:
  X509v3 Subject Key Identifier: 
      A1:*:*:*...
  X509v3 Authority Key Identifier: 
      keyid:1D:*:*:*

  X509v3 Basic Constraints: critical
      CA:TRUE, pathlen:0
  X509v3 Key Usage: 
      Digital Signature, Certificate Sign, CRL Sign

Server certificate has the following in its X509v3 extensions section,

X509v3 extensions:
  X509v3 Basic Constraints: 
      CA:FALSE
  Netscape Cert Type: 
      SSL Client
  X509v3 Key Usage: 
      Digital Signature, Non Repudiation, Key Encipherment
  X509v3 Subject Key Identifier: 
      E1:*:*...
  X509v3 Authority Key Identifier: 
      keyid:F9:*:*:*...

  X509v3 Extended Key Usage: 
      TLS Web Client Authentication
          1.3.*.*.*...: critical
            02.
            TYPE:TEST..FCC_ID:12345..DP_ID:54321
          1.3.*.*.*...: critical
      ..NULL:TRUE

The CA certificate don't have the X509v3 Extended Key Usage section.

Answer 1

Years later this question still exists and has no (satisfying) answer...

[TL;DR]

From the OpenSSL documentation X509_VERIFY_PARAM_set_flags:

[For OpenSSL,] by default, any unhandled critical extensions in certificates [...] results in a fatal error.

In Node.js HTTPS/TLS Server, a workaround can be to set the rejectUnauthorized option to false. If the client certificate is rejected (checked from req.client.authorized during request), we can manually re-check (may be with pkijs, node-forge or other x509 certificate aware packages), ignoring offending extensions.

---

The long journey to come to this conclusion:

In Node.js HTTPS/TLS Server, the option controlling the client certificate verification is requestCert. Tracked down to 'crypto_tls.cc' file, this option is used by SSL_set_verify():

// extracted from 'src/crypto/crypto_tls.cc', for 'requestCert' = true
void TLSWrap::SetVerifyMode(const FunctionCallbackInfo<Value>& args) {
    // ...
    verify_mode = SSL_VERIFY_PEER;
    // ...
    SSL_set_verify(wrap->ssl_.get(), verify_mode, VerifyCallback);
}

The certificate validation will be done in crypto_common.cc. The function SSL_get_peer_certificate will call VerifyCallback.

// extracted from 'src/crypto/crypto_common.cc'
long VerifyPeerCertificate(  // NOLINT(runtime/int)
  const SSLPointer& ssl,
  long def) {  // NOLINT(runtime/int)
    long err = def;  // NOLINT(runtime/int)
    if (X509* peer_cert = SSL_get_peer_certificate(ssl.get())) {
        X509_free(peer_cert);
        err = SSL_get_verify_result(ssl.get());
        // ...
        }
    return err;
}

From the OpenSSL documentation SSL_set_verify:

The verify_callback function is used to control the behaviour when the SSL_VERIFY_PEER flag is set. It must be supplied by the application and receives two arguments: preverify_ok indicates, whether the verification of the certificate in question was passed (preverify_ok=1) or not (preverify_ok=0). x509_ctx is a pointer to the complete context used for the certificate chain verification.

But in Node.js 'crypto_util.cc' file:

// extracted from 'src/crypto/crypto_util.cc'
int VerifyCallback(int preverify_ok, X509_STORE_CTX* ctx) {
    return 1;
}

So:

• we can't tweak the verification flags (no JS option and we shouldn't use the X509_V_FLAG_IGNORE_CRITICAL flag in production),
• we can't retrieve error information from ctx to see which error occurred (no JS callback).

---

Note: quotes from Node.js version 13.18.0 and OpenSSL version 3.1.2