Im trying to connect a tensor flow model to the app and use the camera as input. The camera works, but when I attach the frame processor created following the documentation, an error gets thrown every frame:
ERROR Camera.onError(unknown/unknown): [unknown/unknown] Compiling JS failed: 1:1:invalid empty parentheses '( )' Buffer size 3 starts with: 280a29 [unknown/unknown: [unknown/unknown] Compiling JS failed: 1:1:invalid empty parentheses '( )' Buffer size 3 starts with: 280a29]
This is my App.tsx.
export default function App() {
// States
const [loaded, setLoaded] = useState<boolean>(false);
const [model, setModel] = useState<TensorflowModel>();
const { hasPermission, requestPermission } = useCameraPermission();
const { resize } = useResizePlugin();
// request camera permission
while (!hasPermission) requestPermission();
const device = useCameraDevice("back");
// load tensor flow model
useEffect(() => {
loadTensorflowModel(require("./assets/products.tflite")).then(
(tensorFile) => {
setModel(tensorFile);
setLoaded(true);
}
);
}, []);
const frameProcessor = useFrameProcessor(
(frame) => {
"worklet";
const data = resize(frame, {
scale: {
width: 224,
height: 224,
},
pixelFormat: "rgb",
dataType: "uint8",
});
const output = model!.runSync([data]);
console.log(`Detected: ${output}`);
},
[]
);
if (device == null || !loaded || model == undefined)
return <View />; // TODO: add loading screen
else
return (
<SafeAreaView style={styles.root}>
<CameraPage device={device} frameProcessor={frameProcessor} />
</SafeAreaView>
);
}
and the :
<Camera
style={StyleSheet.absoluteFill}
device={device}
frameProcessor={frameProcessor}
isActive={true}
/>
I am testing on my Galaxy S21 btw.
I tried moving the frame processor to its own file, thinking maybe the model being loaded twice was affecting it, but the same thing.