create llamaindex servicecontext without embedding model?

176 Views Asked by At

creating vector stores is expensive and only needs a model during generation. So why do the docs say we need to include the same model used for creation during load of an existing db? That incurs the loading of large models unnecessarily.

I would like to rewrite the following working code (snippet):

...
embed_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-base-en-v1.5", max_length=512
)
service_context = ServiceContext.from_defaults(embed_model=embed_model, llm=None)

if exists:
    vector_store = FaissVectorStore.from_persist_dir(persist_dir)
    storage_context = StorageContext.from_defaults(
        vector_store=vector_store,
        persist_dir=persist_dir,
    )
    index = load_index_from_storage(
        storage_context=storage_context, service_context=service_context
    )
else:
    dim = 768
    faiss_index = faiss.IndexFlatL2(dim)
    vector_store = FaissVectorStore(faiss_index=faiss_index)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)
    documents = _get_documents()
    index = VectorStoreIndex.from_documents(
        documents, storage_context=storage_context, service_context=service_context
    )
    index.storage_context.persist(persist_dir=persist_dir)

To something like this:

dim = 768
if exists:
    vector_store = FaissVectorStore.from_persist_dir(persist_dir)
    storage_context = StorageContext.from_defaults(
        vector_store=vector_store,
        persist_dir=persist_dir,
    )
    index = load_index_from_storage(storage_context=storage_context, dim=dim)
else:
    embed_model = HuggingFaceEmbedding(
        model_name="BAAI/bge-base-en-v1.5", max_length=512
    )
    service_context = ServiceContext.from_defaults(
        embed_model=embed_model, llm=None
    )
    faiss_index = faiss.IndexFlatL2(dim)
    vector_store = FaissVectorStore(faiss_index=faiss_index)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)
    documents = _get_documents()
    index = VectorStoreIndex.from_documents(
        documents, storage_context=storage_context, service_context=service_context
    )
    index.storage_context.persist(persist_dir=persist_dir)

I also created an issue on github for this here, but wanted more eyeballs on the issue ;)

0

There are 0 best solutions below