I have browser extension that works perfectly in chrome.
Extension do the following:
- It makes monkey patch for
navigator.mediaDevices.getUserMediafunction. Patch replaces media stream from camera with media stream from canvas. Canvas has rendering of human face with webGL. - Content script starts capturing audio stream from microphone with
navigator.mediaDevices.getUserMedia. Then this stream goes to AudioWorkletNode for some simple pre-processing. After that data goes from content script to background where it is processed with TensorFlow's networks. The result (the instruction of how to open the mouth of face model) sends back to content script and applies to rendering process.
While migrating I faced problems:
Difficult to restart
Close extension app
Go to Safari->Settings-Extensions and press uninstall.
In folder that appears right-click on my app and press "Move to trash"
Install application again (Press play button in Xcode)
Go to Safari->Settings-Extensions and press "Clear storage" button.
Restart safari.
Only after this magic procedure the modified extension works correctly. In chrome I only need to press refresh button on extensions card, and refresh the page where I test my extension.
The first question is: Is there any way to update modification in extension simpler way?
Service worker doesn't support webGL
Tensorflow uses webgl for computations, but it is not allowed to use webgl in service worker in safari.
What can I do with it?