Sign HLKX package with key stored in HSM

Question

I need to sign a HLKX package created with Hardware Lab Kit using a certificate/key stored in a HSM. I do have the certificate as a file, but it contains only the public key and the private key is stored in the HSM which is not connected to the computer the package needs to be signed on.

Using the Sign function provided in the code sample here: https://learn.microsoft.com/en-us/windows-hardware/test/hlk/user/hlk-signing-with-an-hsm I get the exception: System.Security.Cryptography.CryptographicException: 'Cannot locate the selected digital certificate.'

I assume this is caused because it does not have the private key that belongs to the certificate available. There is some vague information about CSP dlls that provide the actual signing functionality, but I could find any information on how its interface would have to look like - also how to tell the system for which certificates this CSP dll is responsible for. Also, the HSM is connected to Hashicorp Vault and not used directly.

In case of signing binaries using signtool.exe it was easy to create the digest in one call then use that digest in custom code to get the signature and call signtool.exe again to actually add the signature. But it is unclear to me how this is done when signing the HLKX package. Is there any (useful) documentation or code sample on how to achieve that?

Answer 1

Here is how signature works:

1. Get a binary somehow. Build from source for example. Can be anything, but its format matters.
2. Compute a hash of the binary
3. Compute a digital signature of the hash, using an asymmetric algorithm like RSA. Whatever thing does this step needs direct access to the private key (not the public key nor the certificate that holds it).
4. Embed the signature in the right binary format, and usually also the hash and certificate in the binary.

Steps 1 and 2 are well known. It gets complicated at step 3 and 4.

If your private key is in an HSM, only the HSM can do step 3: sign the hash.

The fact that Hashicorp Vault (Enterprise version) is connected to the HSM does not help because Vault does not provide a secret engine that can do step 4. Vault does not know how to insert a signature in a HLK binary file. Even worse, Vault, as of July 2022, does not offer a way to sign an arbitrary hash with a private key that is stored in an HSM.

Long story short: you cannot use Vault for your use case.

Since only the HSM has access to the private key, the HSM will do the actual signature. It will never reveal the private key, only return the results to the caller. Whatever called the HSM will have to insert the signature in the package, respecting the format.

Using signtool.exe is out of the question because it does not support the Open Packaging Conventions standard. But it will help you debug your configuration.

I would break the problem down to these steps:

Make the HSM work with Windows.

Read your HSM vendor documentation so that your HSM's DLL is installed and configured to work as a 1st class Windows Cryptographic Service Provider. It could a smart card inserted in your workstation or a networked HSM (via a proprietary protocol, beware of firewalls between you and it).

Import your signing certificate in your Certificate Store

Using the The Hardware Lab Kit to sign your .hlkx package, you must select the "Use the certificate store" option. The "certificate file" option implies that you have the private key, but you don't. The HSM has it and it will never export it.

So for that option to work, the certificate must be in your certificate store. Check your vendor documentation to make sure you import the certificate in a way that tells the certificate store that the private key is held by the HSM.

Some examples have you provide the certificate file, but I suspect it's just to extract its hash and look it up in the certificate store.

Test your configuration

At this point, you need a referee. You don't want to debug your HSM configuration when a pesky bug in your code is the real culprit. So run signtool to sign any old binary you have lying around. You can sign notepad.exe if you want:

copy %windir%\system32\notepad.exe .\my-notepad.exe
signtool sign /f certificate.cer /csp "Hardware Cryptography Module" my-notepad.exe

Replace Hardware Cryptography Module with the name of your HSM's CSP name. Check your vendor documentation for this.

Run the code

There is a sample C# program that will sign using an HSM. You must provide the CSP's name, it is the same as the one you provided when testing with signtool.exe.

Answer 2

In order to use the PackageDigitalSignatureManager approach as described here: https://learn.microsoft.com/en-us/windows-hardware/test/hlk/user/hlk-signing-with-an-hsm two preconditions need to be addressed:

1. A cryptographic service provider dll needs to be created and registered
2. The certificate needs to be imported to the certificate store and linked to the provider.

Regarding 1: There is a sample project (in C++ to create a provier dll and a command line executable that is handling the registering/unregistering of the dll). For the registration to work this way, the DLL has to be present in the Windows\System folder. The source sample can be found here: https://www.microsoft.com/en-us/download/details.aspx?id=30688

Regarding 2: I found a way to link the certificate with the provider and posted the C# code to do so in my other question here: How to link KSP DLL to Certificate

Remarks: Doing it this way worked for signing hlkx files from within Hardware Lab Kit software or by signing manually using the PackageDigitalSignatureManager class as described in the example linked above. For me it does not (yet) work when trying to use the certificate when signing executables using Microsoft's signtool.exe. It always complains about the private key not being available for the certificate, although the certificate in the store also states that the private key is available. But as I was focusing on hlkx signing I wasn't looking into this issue (yet).