Embedded command line tool called via Runtime.exec in a Java sandboxed app on MacOS results in signal 4

Question

Objective

I want to ship my java app myapp to macOS appstore, the app uses a command line tool mytool called with:

Runtime.getRuntime().exec(arrayOf("mytool", "argument")

Tools

• Mac M1Pro with Ventura 13.1
• OpenJDK 18.0.2 (x64)
• Xcode 14.2

Problem

When I try to run it it crashes with the error:

Cannot run program "/Applications/MyApp.app/Contents/app/mytool": error=0, Failed to exec spawn helper: pid 64709, signal: 4

What I have done so far

After reading:

• Embedding a command-line tool in a sandboxed app
• Sign the Application Package (macOS)

My understanding is that I have to:

• sign my command line tool mytool with entitlements:

<key>com.apple.security.app-sandbox</key>
<true/>
<key>com.apple.security.inherit</key>
<true/>

• place that tool inside the .app image and then sign the image

To achieve that I do:

/usr/bin/codesign --timestamp --options runtime \
-s "3rd Party Mac Developer Application: Radoslaw Juszczyk (*********)" \
-i com.rjuszczyk.myapp.mytool \
--entitlements mytool.plist \
-vvvv inputDir/mytool

to sign mytool, and then I place it in the inputDir

Then I copy my myapp.jar to the same inputDir and execute:

jpackage --name MyApp
--input inputDir
--mac-sign --mac-signing-key-user-name "Radoslaw Juszczyk"
--mac-package-signing-prefix com.rjuszczyk.myapp.
--mac-app-store
--type pkg
--mac-entitlements myapp.plist
--main-jar myapp.jar

Now it creates MyApp.pkg successfully, which passes verification in the Transporter.app

When I install it, it installs correctly, and the installed .app has this hierarchy:

When the execution hits Runtime.exec it crashes with the mentioned above error.

I tried also to specify different launch mechanisms (posix_spawn, fork) in the jpackage tool :

--java-options "-Djdk.lang.Process.launchMechanism=posix_spawn"

but no luck. My understanding of the problem is that somehow the process started with Runtime.exec does not start in the sandbox, and since mytool requires to be inside sandbox it fails to execute (I am not sure if it makes any sense).

My entitlement files for reference:

myapp.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.security.cs.allow-jit</key>
    <true/>
    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
    <true/>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
    <key>com.apple.security.cs.allow-dyld-environment-variables</key>
    <true/>
    <key>com.apple.security.cs.debugger</key>
    <true/>
    <key>com.apple.security.device.audio-input</key>
    <true/>
    <!-- Add additional entitlements here, for example for network or hardware access. -->
</dict>
</plist>

mytool.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.security.inherit</key>
    <true/>
</dict>
</plist>

This is the moment when I feel like I run out of options. The only thing which maybe is causing that (but I do not believe it matters) is the fact that I place my mytool under Contents/app instead of Contents/MacOS but I am not sure how make the jpackage tool to place it somewhere else.

I hope there is someone who went through the same process and can point out what I am missing. Or point me to some docs describing how to do it.

Answer 1

Couple of suggestions, especially when building with Xcode if you are also targeting mobile apps, I see you signed the entitlements but not the deepsign.

maybe use -c --verbose options to get more debug information to help you out.

---

Try creating a new text file named java.entitlements with the following contents:

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>com.apple.security.app-sandbox</key>
        <true/>
        <key>com.apple.security.inherit</key>
        <true/>
    </dict>
</plist>


// Now grant these entitlements to the java executable:
codesign -f -s "Mac Developer: John Smith (XYZ)" --entitlements "java.entitlements" "Home/bin/java"

Make sure to use your own developer certificate with the correct paths appropriate to your situation.

Because this is a cryptographic operation, complete changes to the Info.plist prior to this step.

Now Configure Tools / for e.g. in Xcode Make sure that your application or framework target during build

• first step will copy
• second step sign this PlugIn bundle when you build. Do this by adding a "New Copy Files Phase" to the target. Rename it to “Copy Java Runtime”), choose PlugIns as the destination, choose the Java.runtime directory, and ensure that code sign on copy is selected.

Sign the Java.runtime bundle Sometimes code sign on copy will complain that the Java.runtime bundle is not already signed, so you may have to manually deep sign the bundle in order to have your tool or Xcode re-sign it when building your project:

codesign --deep -f -s "Mac Developer: John Smith (XYZ)" Java.runtime/

---

Ultimately I think it is in the codesign look at this ref. issue on github to fix it Someone forced it to code sign to run it ...

codesign --force --deep --sign - ./Test-signed.app

This is for when a process fails to start on macos similar to yours https://github.com/libgdx/packr/issues/161