Scene Kit Rotation - rotating around X and Y axis only, causing Z rotation

Question

I am trying to control the orientation of a box in Scene Kit (iOS) using gestures. I am using the translation in x and y to update the x and y rotation of the SCNNode.

After a long search I have realised that x and y rotation will always lead to z rotation, thanks to this excellent post: gamedev stack exchange: I'm rotating an object on two axes, so why does it keep twisting around the third axis?

So I am trying to get the z rotation causes, and then remove this from my object by applying the inverse quaternion however when I rotate the object 90 deg around x, and then 90 deg around Y it behaves VERY weirdly.

It is almost behaving as it is in gimbal lock, but I did not think that using quaternion in the way that I am would cause gimbal lock in this way.

I am sure it is something I am missing, or perhaps I am not able to remove the z rotation in this way.

Thanks!

The code I am using to rotate during pan gesture is:

@objc func panGestureQ(_ gestureRecognize: UIPanGestureRecognizer) {
            guard let boxNode = boxNode else { return }
            
            let translation = gestureRecognize.translation(in: gestureRecognize.view!)
            let angleY = Float(translation.x) * 0.003
            let angleX = Float(translation.y) * 0.003
            
            let rotationX = simd_quaternion(angleX, simd_float3(x: 1, y: 0, z: 0))
            let rotationY = simd_quaternion(angleY, simd_float3(x: 0, y: 1, z: 0))
            
            let combined = rotationX * rotationY
            
            let tempNode = SCNNode()
            tempNode.simdOrientation = boxNode.simdOrientation
            let zBefore = tempNode.eulerAngles.z
            
            tempNode.simdOrientation =  combined * tempNode.simdOrientation
            
            let zAfter = tempNode.eulerAngles.z
            let zDiff = zAfter - zBefore
            
            let rotationZ = simd_quaternion(zDiff, simd_float3(x: 0, y: 0, z: 1))

            boxNode.simdOrientation =  rotationZ.inverse * combined * boxNode.simdOrientation
            
            gestureRecognize.setTranslation(CGPoint.zero, in: gestureRecognize.view)
            
           
        }

I have added a video of the strange behaviour here

And the code example is here