We're creating a video calling app for the iPhone and it's working really well and on iOS17 we can even use gestures to make the video overlay animated reactions on the video which is really nice. But that came for free without any code from us.
I was wondering if anyone knows if it's possible to access information about what reactions were sent by the user?
i.e. if a user does a two thumbs up then it will show fireworks in the video.
Can I intercept that in code in my app and send some data somewhere in response to the fireworks shown?
For reference... these are the reactions I am referring to... https://www.macrumors.com/how-to/use-reaction-effects-facetime/
You can detect which gesture effects are enabled, and whether or not they are enabled, and can trigger them programmatically, using these APIs:
https://developer.apple.com/documentation/avfoundation/avcapturedevice/system_video_effects_and_microphone_modes
Specifically, you may be able to detect changes in this property to see when gesture effects have been triggered:
https://developer.apple.com/documentation/avfoundation/avcapturedevice/4248897-reactioneffectsinprogress
Your app can opt-in to gesture effects using the
NSCameraReactionEffectsEnabledInfo.plist key. You may also be able to disable them app-wide by setting it to<false/>, which is what I am interested in doing. I will be testing that shortly; I will post my result in a comment.