Yubico private key certificate signing no longer works with openssl3

Question

I store my PKI CA certificate and private key on a Yubikey and used it to issue end user certificates but after upgrading to openssl3 from openssl1 this no longer works.

The script that I have used to sign certificate requests which works with openssl1 but no longer works with openssl3:

pki_path=~/pki/paul
pin=$(cat ~/yubico/pin.txt)
echo "Yubico PIN: $pin"
openssl engine dynamic -pre SO_PATH:/usr/lib/engines-3/pkcs11.so -pre ID:pkcs11 -pre NO_VCHECK:1 -pre LIST_ADD:1 -pre LOAD -pre MODULE_PATH:/usr/lib/opensc-pkcs11.so -pre VERBOSE
openssl x509 -engine pkcs11 -CAkeyform engine -CAkey slot_0-id_2 -sha384 -CA $pki_path/ca.crt -req -passin pass:$pin -in $pki_path/reqs/$2.req -extfile x509-types/$1 -days 365 -out $pki_path/issued/$2.crt
cat $pki_path/issued/$2.crt $pki_path/ca.crt > $pki_path/bundle/$2.crt

I have updated the pkcs11 path but everything else is the same, running the commands by hand it works to register the engine but attempting to sign fails.

λ ~/pki/scripts/ openssl engine dynamic -pre SO_PATH:/usr/lib/engines-3/pkcs11.so -pre ID:pkcs11 -pre NO_VCHECK:1 -pre LIST_ADD:1 -pre LOAD -pre MODULE_PATH:/usr/lib/opensc-pkcs11.so -pre VERBOSE
(dynamic) Dynamic engine loading support
[Success]: SO_PATH:/usr/lib/engines-3/pkcs11.so
[Success]: ID:pkcs11
[Success]: NO_VCHECK:1
[Success]: LIST_ADD:1
[Success]: LOAD
[Success]: MODULE_PATH:/usr/lib/opensc-pkcs11.so
[Success]: VERBOSE
Loaded: (pkcs11) pkcs11 engine
λ ~/pki/scripts/ openssl x509 -engine pkcs11 -CAkeyform engine -CAkey slot_0-id_2 -sha384 -CA ~/pki/paul/ca.crt -req -passin pass:$pin -in ~/pki/paul/reqs/paul.csiki.req -extfile x509-types/client -days 365 -out ~/pki/paul/issued/paul.csiki.crt
Engine "pkcs11" set.
Certificate request self-signature ok
subject=CN = paul.csiki
Failed to enumerate slots
PKCS11_get_private_key returned NULL
Could not read CA private key from org.openssl.engine:pkcs11:slot_0-id_2
409754AB157F0000:error:40000067:pkcs11 engine:ERR_ENG_error:invalid parameter:eng_back.c:603:
409754AB157F0000:error:13000080:engine routines:ENGINE_load_private_key:failed loading private key:crypto/engine/eng_pkey.c:79:

Package versions:

opensc 0.23.0-1
openssl 3.0.9-1

pkcs11-tool seems to be able to pull the certificate and private key in the correct slot with the correct id.

λ ~/ pkcs11-tool --module /usr/lib/opensc-pkcs11.so --login -O
Using slot 0 with a present token (0x0)
Logging in to "Alexandru Paul Csiki CA 2022".
Please enter User PIN:
Private Key Object; RSA
  label:      SIGN key
  ID:         02
  Usage:      sign
  Access:     always authenticate, sensitive, always sensitive, never extractable, local
Public Key Object; RSA 2048 bits
  label:      SIGN pubkey
  ID:         02
  Usage:      verify
  Access:     none
Certificate Object; type = X.509 cert
  label:      Certificate for Digital Signature
  subject:    DN: CN=Alexandru Paul Csiki CA 2022
  serial:     D9A92431209229505AAFE58D45432A67
  ID:         02
λ ~/ pkcs15-tool -D
Using reader with a card: Yubico YubiKey OTP+FIDO+CCID 00 00
PKCS#15 Card [Alexandru Paul Csiki CA 2022]:
    Version        : 0
    Serial number  : 00000000
    Manufacturer ID: piv_II
    Flags          :

Private RSA Key [SIGN key]
    Object Flags   : [0x01], private
    Usage          : [0x04], sign
    Access Flags   : [0x1D], sensitive, alwaysSensitive, neverExtract, local
    Algo_refs      : 0
    ModLength      : 2048
    Key ref        : 156 (0x9C)
    Native         : yes
    Auth ID        : 01
    ID             : 02

Public RSA Key [SIGN pubkey]
    Object Flags   : [0x00]
    Usage          : [0xC0], verify, verifyRecover
    Access Flags   : [0x02], extract
    ModLength      : 2048
    Key ref        : 156 (0x9C)
    Native         : yes
    ID             : 02
    DirectValue    : <absent>

X.509 Certificate [Certificate for Digital Signature]
    Object Flags   : [0x00]
    Authority      : no
    Path           :
    ID             : 02
    Encoded serial : 02 11 00D9A92431209229505AAFE58D45432A67

I have tried to use the token uri instead of the slot + id to no avail too.

Answer 1

Check if the error (PKCS11_get_private_key returned NULL) is related to OpenSC/OpenSC issue 206

The version of the opensc-pkcs11.so file being referenced by your OpenSSL command may be outdated. Meaning it uses an older version of the opensc library that does not correctly handle the SPKI (Subject Public Key Info), it would not the SPKI properly. The SPKI is used to represent the public key and its associated algorithm and parameters.

In your script, you are specifying the path to the opensc-pkcs11.so file as part of the OpenSSL command:

openssl engine dynamic -pre SO_PATH:/usr/lib/engines-3/pkcs11.so -pre ID:pkcs11 -pre NO_VCHECK:1 -pre LIST_ADD:1 -pre LOAD -pre MODULE_PATH:/usr/lib/opensc-pkcs11.so -pre VERBOSE

If the opensc-pkcs11.so file at /usr/lib/opensc-pkcs11.so is from an older version of OpenSC, it may not be compatible with the new OpenSSL 3.0 version, causing the error you are seeing.

To resolve this, you might need to ensure that the opensc-pkcs11.so file being referenced is from a compatible version of OpenSC. This could involve updating OpenSC to a newer version, or building a new opensc-pkcs11.so file from the updated OpenSC source code.
After doing so, you would adjust your OpenSSL command to reference the path of the updated opensc-pkcs11.so file.

Answer 2

im getting the same issue but realized a few things. TLDR i added a config file when executing the x509 command and that solved my issue.

1). download the latest opensc from brew

brew install libp11
brew install opensc

3). link things directly in that library not to get confused with what's currently in /usr/local/lib 4). permission issues with the files might be there as well (to test run as sudo if you are getting permission issues).

(my openssl3 was installed by macports which is why it's at that "bizzare" location in opt).

sudo  OPENSSL_CONF=engine.conf /opt/local/bin/openssl-3 x509 -engine pkcs11 -CAkeyform engine -CAkey slot_0-id_2 -sha256 -CA intermediate-ca-certificate.pem -req -days 731 -in client-csr.pem -out END-certificate.pem -extfile client.conf -extensions x509v3

my engine.conf file reads as:

    openssl_conf = openssl_init

    [openssl_init]
    engines = engine_section

    [engine_section]
    pkcs11 = pkcs11_section

    [provider_sect]
    legacy = legacy_sect

    [legacy_sect]
    activate = 1 

    [pkcs11_section]
    engine_id = pkcs11
    #dynamic_path is not required if you have installed
    #the appropriate pkcs11 engines to your openssl directory
    dynamic_path = /usr/local/Cellar/libp11/0.4.12/lib/engines-3/pkcs11.dylib
    MODULE_PATH = /usr/local/Cellar/opensc/0.23.0/lib/opensc-pkcs11.so
    init = 1