Can't load an .onnx file in my vite react project

Question

New to webdev and I'm trying to use an onnx (using onnxruntime-web) file in my react project but vite is complaining.

1:pytorch2.0.0:���'
                          ^
2: �

: input
error during build:
Error: Parse error @:1:24

I was able to get it working using webpack but don't understand why it wont work with vite. I know the rest of my code works since it was working with webpack.