react-native-vision-camera - Attaching frame processor to the camera throws errors

93 Views Asked by At

Im trying to connect a tensor flow model to the app and use the camera as input. The camera works, but when I attach the frame processor created following the documentation, an error gets thrown every frame:

ERROR Camera.onError(unknown/unknown): [unknown/unknown] Compiling JS failed: 1:1:invalid empty parentheses '( )' Buffer size 3 starts with: 280a29 [unknown/unknown: [unknown/unknown] Compiling JS failed: 1:1:invalid empty parentheses '( )' Buffer size 3 starts with: 280a29]

This is my App.tsx.

export default function App() {
  // States
  const [loaded, setLoaded] = useState<boolean>(false);
  const [model, setModel] = useState<TensorflowModel>();

  const { hasPermission, requestPermission } = useCameraPermission();
  const { resize } = useResizePlugin();

  // request camera permission
  while (!hasPermission) requestPermission();
  const device = useCameraDevice("back");

  // load tensor flow model
  useEffect(() => {
    loadTensorflowModel(require("./assets/products.tflite")).then(
      (tensorFile) => {
        setModel(tensorFile);
        setLoaded(true);
      }
    );
  }, []);

  const frameProcessor = useFrameProcessor(
    (frame) => {
      "worklet";
      const data = resize(frame, {
        scale: {
          width: 224,
          height: 224,
        },
        pixelFormat: "rgb",
        dataType: "uint8",
      });
      const output = model!.runSync([data]);
      console.log(`Detected: ${output}`);
    },
    []
  );


  if (device == null || !loaded || model == undefined)
    return <View />; // TODO: add loading screen
  else
    return (
      <SafeAreaView style={styles.root}>
        <CameraPage device={device} frameProcessor={frameProcessor} />
      </SafeAreaView>
    );
}

and the :

        <Camera
          style={StyleSheet.absoluteFill}
          device={device}
          frameProcessor={frameProcessor}
          isActive={true}
        />

I am testing on my Galaxy S21 btw.

I tried moving the frame processor to its own file, thinking maybe the model being loaded twice was affecting it, but the same thing.

0

There are 0 best solutions below