Issue On Writing Faiss Index to a File when using through Docker Image

78 Views Asked by At

I have a FastAPI Docker Image where in the startup section I am fetching the binary version of my FAISS index from Redis, unpickling it using pickle.loads and then using

file_path = os.path.join(folder_path, 'index.faiss')
faiss.write_index(faissModelFromRedis,file_path)

to write it to a file. This works in local, but when deployed using Docker to Azure Web App it throws

File "/app/main.py", line 38, in loadModelAndSetRetriever
2024-01-27T14:03:27.547507225Z     faiss.write_index(faiss_model,file_path)
2024-01-27T14:03:27.547510425Z   File "/usr/local/lib/python3.8/site-packages/faiss/swigfaiss_avx2.py", line 10200, in write_index
2024-01-27T14:03:27.547513825Z     return _swigfaiss_avx2.write_index(*args)
2024-01-27T14:03:27.547517125Z TypeError: Wrong number or type of arguments for overloaded function 'write_index'.
2024-01-27T14:03:27.547520225Z   Possible C/C++ prototypes are:
2024-01-27T14:03:27.547523325Z     faiss::write_index(faiss::Index const *,char const *)
2024-01-27T14:03:27.547526526Z     faiss::write_index(faiss::Index const *,FILE *)
2024-01-27T14:03:27.547529526Z     faiss::write_index(faiss::Index const *,faiss::IOWriter *)

How can we resolve this?

I tried to use pickle.dumps but with Faiss, it is unable to read_index of the saved file.

Also tried to add chmod 777 or new user add command in Dockerfile thinking it's a write access issue from faiss.

0

There are 0 best solutions below