I am developing an iOS application for a "green screen" using ARKit. That is display an image as a background in a ARFaceTrackingConfiguration AR session (similar to the "green screen" functionality available in other apps such as zoom). I want the image to remain visible all the time and expand to the edges of the screen regardless of the movement of the device. I am using ARKit and SceneKit to accomplish this. In summary here is what I am doing:
- Add an ARSCNView to the view controller
- Create an SKScene with the same size (expressed in points) as the ARSCNView
- Create an SKSpriteNode and add it as child to the SKScene
- Create an SCNPlane of a specific size (expressed in meters)
- Create an SCNNode and set its geometry to be the the SCNPlane created in the previous step
- Add the SCNNode to the ARCSNView
Complete code to accomplish this is below:
@IBOutlet var arSceneView: ARSCNView!
var virtualBackgroundNode = SCNNode()
...
func addGreenScreenScene() {
let skScene = createSKScene()
let plane = createScnPlane()
virtualBackgroundNode.geometry = plane
let material = SCNMaterial()
material.diffuse.contents = skScene
virtualBackgroundNode.geometry?.materials = [material]
if let cameraPosition = arSceneView.pointOfView?.scale {
let position = SCNVector3(0, 0, cameraPosition.z - 13)
os_log("camera position=\(cameraPosition.x) \(cameraPosition.y) \(cameraPosition.z)")
os_log("arSceneView frame=\(self.arSceneView.frame.width) \(self.arSceneView.frame.height)")
virtualBackgroundNode.position = position
}
arSceneView.pointOfView?.addChildNode(virtualBackgroundNode)
}
func createSKScene() -> SKScene? {
guard let image = getImage() else {
return nil
}
let skScene = SKScene(size: CGSize(width: arSceneView.frame.width,
height: arSceneView.frame.height))
let texture = SKTexture(image: image)
let spriteNode = SKSpriteNode(texture:texture)
spriteNode.position = CGPoint(x: skScene.size.width / 2.0,
y: skScene.size.height / 2.0)
spriteNode.size = skScene.size
spriteNode.yScale = -1.0
skScene.scaleMode = .aspectFit
skScene.addChild(spriteNode)
return skScene
}
func createScnPlane() -> SCNPlane {
// TODO: Determine the correct size for this
// such that it covers the entire frame of
// arSceneView.
return SCNPlane(width: 12, height: 18)
}
func getImage() -> UIImage? {
return UIImage(named: photoInfo.imageName)
}
The problem I am having is how to calculate the correct size of the SCNPlane (in method createScnPlane) so that it spans across the entire frame of the ARSCNView.
Disclaimer: I am new to AR development. Is there a different way to develop "green screen" functionality?