How to run/debug open-source macOS `Privileges` app w/ XPC service/daemon and DockTile plugin

Question

I'm attempting to try out some modifications in SAP's Privileges.app. Unfortunately, their (understandable) Support policy is

This project is 'as-is' with no support, no changes being made. You are welcome to make changes to improve it but we are not available for questions or support of any kind.

Unfortunately, this app uses two constructs I've never come across before in my professional experience, an XPC service + helper (Launch daemon?) and a DockTile plugin. I'm having a hard time just fundamentally getting the app to work when launched from Xcode - it launches, but it seems that there are issues between (maybe?) sandboxing, signing and perhaps entitlements? I've updated the signing to use my own team, of course, and everything compiles/links/launches properly, but when the XPC service tries to install the helper tool it fails

2022-06-29 17:03:56.284544-0500 PrivilegesXPC[13079:128535] [logging-persist] cannot open file at line 45530 of [9ff244ce07]
2022-06-29 17:03:56.284570-0500 PrivilegesXPC[13079:128535] [logging-persist] os_unix.c:45530: (0) open(/var/db/DetachedSignatures) - Undefined error: 0
2022-06-29 17:04:21.060214-0500 PrivilegesXPC[13079:128537] SAPCorp: ERROR! Failed to connect to helper tool: NSCocoaErrorDomain / 4097
2022-06-29 17:04:31.471555-0500 Privileges[13064:127420] SAPCorp: ERROR! Error Domain=NSPOSIXErrorDomain Code=25 "Inappropriate ioctl for device"
2022-06-29 17:04:45.717751-0500 Privileges[13064:129162] SAPCorp: ERROR! Installation of the helper tool failed: Error Domain=CFErrorDomainLaunchd Code=4 "(null)"

As near as I can tell, the last two errors are thrown from a failure in

    success = SMJobBless(
        kSMDomainSystemLaunchd,
        CFSTR("corp.sap.privileges.helper"),
        self->_authRef,
        &error
    );

but I haven't been able to ascertain why this is failing. Searching for errors around Inappropriate ioctl for device has not been fruitful, unfortunately.

If there's anyone out there with some experience in dealing with apps using some of these more esoteric moving parts that can share some things to try, I'd be much obliged. Bonus points if there's any way to debug code running in a DockTile plugin - as near as I can tell, it's running in SystemUIServer, but I can't attach to that (even as root) from Xcode.

Answer 1

I think I've sorted out getting this running. Here's a few roadblocks I encountered.

1. SMJobBless has some very particular expectations around code-signing - you'll find references to this in some forum posts and there's a sample project that's also referenced with a utility script - which doesn't run on modern macOS because it's written for Python 2 -- which isn't installed by default anymore and a bit difficult to come by. But, after agonizingly converting Python 2-isms over to Python 3, you'll come to find out that that's not the only thing that's changed, a number of the tools (codesign and otool) don't output the same on ARM64 at which time you'll finally stumble across a kind soul that converted SMJobBless.py ... only to find out that it's not actually needed for this project?! Not sure if it's because the Launch Service is contained in the XPC and not the app, but either way - it seems to not be needed.
2. If you've run Privileges before, it'll have installed it's escalated helper, which will stand in the way of a local Xcode build copying itself over - which matters because of the aforementioned code signing. You'll need to clear away these artifacts

   $ sudo rm -rf /Library/PrivilegedHelperTools/corp.sap.privileges.helper
   $ sudo rm /Library/LaunchDaemons/corp.sap.privileges.helper.plist
   

   Just deleting them isn't enough, it seems some sort of runtime launchd state needs to be wiped. It's unclear to me if some incantation of launchctl will clear this out, maybe an invocation of launchctl kickstart -k <foo> or something? I ended up rebooting and that seemed to do the trick anyway.
3. It seems like you need a particular signing certificate to allow the various signing validations that SMJobBless and the XPC communications are doing to be valid. Particularly, it seems you'll need a Developer ID Application, which happens to match what's encoded in the .xcodeproj pulled down from the GitHub repo. This means you can't enable Automatically manage signing as you won't get this type of certificate (as near as I can tell - please correct me if I'm wrong).
4. Once you've got all that sorted, since you aren't signing with the SAP developer's certificate, your certificate will have a different unique Team ID, so you'll need to update SMAuthorizedClients and SMPrivilegedExecutables, respectively, (look for 7R5ZEU67FQ and replace with your team ID) in

   PrivilegesHelper/PrivilegesHelper-Info.plist
   PrivilegesXPC/Info.plist
   

I think that's basically got it. Hope that helps someone else