Get World Coordinates of CGPoint in ARSCNView (Swift)

Question

Problem

I'd like the following function to return to me the world coordinates of a point handled by a tap gesture. I've properly implemented it (using a few online resources) to work with the center point of the frame, but I am having trouble shifting this point around the screen.

---

Attempt

Here is the fully commented and working code for getScreenCoordinates, which returns a tuple containing (0) the direction vector of the screen coordinates and (1) the position vector of the screen coordinates. The issue is having this work with a point other than the center, passed in through the screenCoord parameter.

private func getScreenCoordinates(screenCoord:CGPoint) -> (SCNVector3, SCNVector3) { // (direction, position)
      if let frame = self.canvasView.session.currentFrame {
            let mat = SCNMatrix4(frame.camera.transform) // 4x4 transform matrix describing camera in world space
            let direction = SCNVector3(-1 * mat.m31, -1 * mat.m32, -1 * mat.m33) // orientation of camera in world space
            let position = SCNVector3(mat.m41, mat.m42, mat.m43) // location of camera in world space

            return (dir, position)
      }
      return (SCNVector3(0, 0, -1), SCNVector3(0, 0, -0.2))
}

---

Desired Solution

Provide a modified getScreenCoordinates function that returns the same data type, except instead of getting the world coordinates of the center point of the camera frame, it gets the world coordinates for screenCoord.

Answer 1

maybe the code can help you.

// Get transform using ARCamera
func getCameraTransform(for camera: ARCamer) -> MDLTransform {
  return MDLTransform(transform: camera.transform)
}

// Intercept touch and place object 
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
  guard let touch = touches.first else { return }
  guard touch.tapCount == 1 else { return }

  guard let camera = sceneView.session.currentFrame?.camera else { return }
  let transform = getCameraTranform(for: camera)
  let boxGeometry = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
  let cube = SCNNode(geometry: boxGeometry)
  let position = SCNVector3(
   transform.translation.x,
   transform.translation.y,
   transform.translation.z,
  )
  cube.position = position
  sceneView.scene.rootNode.addChildNode(cube)
}

To see more detail, please see this post https://arvindravi.com/arkit-a-noobs-guide-part-three/