install4j stuck at password input during javafx application code-signing for Mac

Question

Despite all our attempts, we could not get past the password entry popup. Usually, this is an issue with an incorrect password or a "head-space and timing issue", but we've confirmed the password to the KeyStore App is correct.

We are attempting to sign our javafx 17 application in order to submit it to the mac app store. We've followed the instructions from Apple to create the certificate, and from the docs on Install4j to upload the certificate. After attempting several times and checking the KeyChain password we still continue to fail - see image below. I've also allowed full access to the certificate in the keyChain app by double clicking the certificate -> Access Control and checking "Allow all applications to access the item". Despite the certificate being unrestricted, we could not get past the password entry.

Install4j Docs on code-signing: https://www.ej-technologies.com/resources/install4j/v/8.0/help/doc/concepts/codeSigning.html

Preparing your app for distribution: https://developer.apple.com/macos/distribution/

Signing you apps for GateKeeper -- Apple id for apps distributed outside the Mac Store: https://developer.apple.com/developer-id/

Has anyone had a similar issue with Install4j 9.0.7? What was the solution?

--update:

Upon further inspection, the EJ Technologies Install4j docs describe that the certificate "Subject Name" must be "Developer ID Application". I am not sure if this is an accurate description. The certificate's Subject Name, Common Name is "Developer ID Application:MyDeveloperName(123.....)". There is no option from apple to give the Certificate a Subject Name. I'm also not sure if this is related to the password problem.

Answer 1

If you have created the certificate signing request with a private key that uses elliptic curve cryptography, it will not work in 9.0.7. Contact [email protected] for a build that will work in this case. Also, please make sure to select both the public and the private key before exporting from the KeyChain app.

I've also allowed full access to the certificate in the keyChain app by double clicking the certificate -> Access Control and checking "Allow all applications to access the item"

Don't do that, install4j does not access the KeyChain, it works with the PKCS#11 keystore that you export from it and you set a password during export for full access.

The certificate's Subject Name, Common Name is "Developer ID Application:MyDeveloperName(123.....)". There is no option from apple to give the Certificate a Subject Name. I'm also not sure if this is related to the password problem.

No, that is fine, "Developer ID Application" is the type of the certificate.

However, regarding:

We are attempting to sign our javafx 17 application in order to submit it to the mac app store.

This will not work with a "Developer ID Application" certificate which is only suitable for standalone distribution. Submitting to the App Store will be supported in the upcoming install4j 10 (very soon). You need different certificates for that and it is not possible to create a signed artifact with install4j 9 that will be accepted by the App Store.

Answer 2

-- Solution:

Based on the answer provided by Ingo Kegel we were successful in 1) bundling our application into a DMG, 2) signing it, and 3) getting it notarized by Apple using Install4j.

Here is a description of what worked.

1. There are multiple passwords needed. We further had problems with other passwords. After Ingo's comments we were able to get through these. Note that the first password is NOT the apple "Keychain Access" app password. It is the password that you set when you create the .pkcs12 file.

2. This post is a supplement to the instructions provided by EJ-Technologies and by Apple in order to show how we solved a few area's that were slightly troubling.

3. Creating a Developer Password with Apple for our application was slightly confusing. After creating the certificateSigningRequest in the "Keychain Access" app, we then needed to create the Application Certificate in the Apple Developer Account (online).

Below: The developer account page, click on the "certificates and id's".

Below: Next page, click on the plus to add a new certificate.

Below: Next page, select the "Developer ID Application, then click continue.

Once Apple had created the certificate we followed the directions to 1) download the cert from apple. It ends with ".cer". 2) install the ".cer" file to your KeyStore app by double clicking on it. This was also problematic and would not install if there was a similar file in the KeyStore App. 3) Export the cert and its related secret to a directory on the hard drive. The certificate and the secret was slightly ambigious. The certificate appears to be a subdirectory of the secret in the KeyStore application. We selected both, and exported them by right clicking on the two.

The instructions were clear from this point forward. However after successfully passing the password entry, we were then faced with a second password entry popup for Apple's notarization. We are creating a .dmg for MacOS 12. Our JavaFX app had to be signed AND notarized in order for our users to avoid "most" of the warnings by Apple.

As mentioned above, install4J asks for the "app specific password" This password had nothing to do with our application, it is an application password between us and install4j.

I created the password in My Apple user account. Note that this is not from the developer account.

In the next screen after clicking here, I simply provided the name "install4j" in the text-entry to describe the password. Also, a Cut and Paste of the password from Apple did not work and caused an error so I pasted the selection from the browser into a text editor and noticed it was wrapped with brackets and several other characters that would make it invalid. Typing the password directly into the install4J text field worked. You will need to save this password for future uploads as well.

After correctly entering these passwords, install4j took about a minute to bundle and sign our application, then sent the dmg to Apple for notorization. Notorization took several minutes. When the operation completed, our dmg was created.

We tested the .dmg by uploading it to our server and downloading it to another device. Everything worked as expected. Apple still warns the user that "the app is not built by Apple", and "it is downloaded from the internet". For the "inexperianced user", this is much easier than without code-signing and notarization.