Sign Code with a YubiHSM over the Network

Jordan

The error code you said changes our troubleshooting direction. The code 0x80070005 translates directly to a standard Windows Access Denied exception. Because your certificate utility successfully verified the signature earlier, it is highly likely that the utility was run from an elevated administrative command prompt, whereas your signing tool is executing within a standard user security context that lacks permission to read the Key Storage Provider pointer. Even though the actual private key safely resides on your network-attached YubiHSM hardware, Windows still creates a local metadata file to route requests to the device, and this local pointer inherits strict administrative security permissions upon creation.

To resolve this access restriction, you need to manually adjust the permissions on the cryptographic binding. You can achieve this by opening the Local Machine Certificate Manager by typing certlm.msc into your Windows Run dialog. Once you navigate to your Personal certificates store and locate the code signing certificate, you can access the security settings by right-clicking the certificate, navigating to All Tasks, and selecting Manage Private Keys. You must explicitly add your standard Windows user account to this security list and grant it Read permissions, which allows the signing tool process to access the hardware pointer without needing full administrative elevation.

If adjusting the certificate security properties does not immediately clear the access denied error, the permission block might be occurring at the YubiHSM registry configuration level instead of the certificate store. The signing tool requires access to the YubiHSM connection parameters and authentication keys stored in your system registry to facilitate the network handshake. You should open the Registry Editor and navigate to the path HKEY_LOCAL_MACHINE\SOFTWARE\Yubico\YubiHSM\KSP to ensure your specific user account has read access to this directory and its subkeys. Granting read access here ensures the background cryptographic process can successfully retrieve the credentials needed to authenticate over the network to your hardware security module.

Hope this helps :)

VP

Hi Jordan,

The error you are encountering stems from an architectural mismatch between the signing tool and the YubiHSM Key Storage Provider rather than a broken key linkage. Because your certificate utility successfully completed the signature test, the 64-bit Windows environment is communicating perfectly with your network-attached YubiHSM. The Windows SDK includes multiple architecture versions of the signing utility, and development environments often default to the 32-bit executable. A 32-bit process is fundamentally incapable of loading the 64-bit Key Storage Provider library required to access the hardware token, which directly triggers the specific failure regarding private key association.

You must ensure you are invoking the native 64-bit version of the signing tool. You can usually find this executable explicitly located at the file path C:\Program Files (x86)\Windows Kits\10\bin\10.0.xxxxx.0\x64\signtool.exe, replacing the xxxxx with your specific SDK build number. Running this specific application ensures architectural parity with the YubiHSM. Additionally, if your certificate is installed in the local computer store rather than your personal user store, you need to append the /sm parameter to your signing command so the tool searches the correct system-level certificate repository instead of your personal user profile.

If utilizing the strict 64-bit executable path does not resolve the issue, a subtle corruption in the Windows Certificate Store bindings may be at fault. Running the command certutil -repairstore my <thumbprint> from an administrative command prompt forces the operating system to forcefully rewrite the logical association between the public certificate and the hardware-backed private key pointer, resolving any hidden mapping conflicts.

Hope this answer brought you some useful information. If it did, please hit "accept answer". Should you have any questions, feel free to leave a comment.

VP

SignTool can use non-exportable private keys in an HSM as long as Windows sees the key through a supported CSP/KSP and the certificate is correctly bound to that key. The error “Could not associate private key with certificate” indicates that, from the CryptoAPI/CNG perspective, the certificate in the store is not actually bound to the YubiHSM key in a way SignTool can use.

From the available information, the supported and documented pattern is:

1. Ensure the certificate with private key is in the Personal store
   • Import the PFX (containing the cert and the HSM-backed key reference) into the Personal store of the signing account, or
   • Use the vendor tooling so that the cert appears in the Personal store with an associated non-exportable private key.
   • After import, certutil -store my should show the certificate with Private key is NOT exportable and successful signature test, which is already the case.
2. Use SignTool against the certificate in the store
   • Use the subject name or thumbprint to let SignTool locate the cert + key via the KSP/CSP:

          signtool sign /v /s MY /n "CertSubjectName" test.exe
     

     or, if using SHA-1 thumbprint:

          signtool sign /v /s MY /sha1 <thumbprint> test.exe
     

   • This is the documented way to sign when the certificate is in the local store and the key is non-exportable/HSM-backed.
3. If the certificate is only in file form
   • When the certificate is not properly installed, SignTool can be pointed at a certificate file and a CSP/key container explicitly:

          SignTool sign /f HighValue.cer /csp "Hardware Cryptography Module" /k HighValueContainer test.exe
     

   • Replace "Hardware Cryptography Module" with the YubiHSM CSP/KSP name and HighValueContainer with the key container name that maps to the YubiHSM key.
   • This pattern is specifically for cases where the cert is not fully installed but the CSP/KSP and key container are known.
4. Verify certificate properties for code signing
   • The certificate must be suitable for code signing:
     • EKU unset or includes Code Signing (OID 1.3.6.1.5.5.7.3.3).
     • Key Usage unset or includes Digital Signature (0x80).
   • If these are missing, SignTool will not treat it as a valid code-signing certificate.
5. Confirm the private key is visible to the KSP/CSP under the signing account
   • The account running SignTool must be able to access the YubiHSM key via the KSP.
   • If the key was created or imported under a different profile or with restricted permissions, adjust permissions according to the HSM vendor’s guidance so that the signing account can use the key.
6. Alternative tools
   • Once the certificate and key are correctly surfaced via the Windows certificate store and CSP/KSP, any tool that uses the standard Windows CryptoAPI/CNG (including SignTool and other Microsoft signing tools) can sign with the HSM-backed key. The key does not need to be exportable; it only needs to be accessible via the CSP/KSP.

If the above conditions are met and the certificate is correctly installed with a non-exportable private key in the Personal store, SignTool should be able to sign using either the /s MY /n or /s MY /sha1 syntax, or by explicitly specifying the CSP and key container.

References:

• Use SignTool to sign a file
• How to sign an app package using SignTool
• Windows Desktop Application Program
• Importing an SPC into a Certificate Store
• Authenticode Signing of Third-party Cryptographic Service Providers (CSPs)