Using Langchain & ChromaDB, I'm building a chatbot (AI assistant) using OpenAI's gpt-3.5-turbo model in order to answer queries based on context derived from relevant content from a large number of documents(in short RAG). The model takes our chat_history and new query to generate a standalone question in order to keep track of the context.
It works well so far, however the problem I notice is when new questions are unrelated to the previous chats, the model still summarizes the question based on past chat_history. This leads to responses which are not correct.
Example:
Human: Tell me something about Docker.
AI: Docker is a containerisation technology …blah blah…..(correct response)
Human: How about Docker Compose?
AI: Docker compose is….blah blah….(correct response)
Human: How about something on enterprise security?
<<AI summarises this question as: Can you provide information on enterprise security, specifically in the context of Docker and Docker Compose? —> This is the problem as the last question is unrelated to Docker.>>
AI: In the context of enterprise security with Docker and Docker Compose, I do not have any information in the provided documents.
I think the question summarizer needs to be tweaked somehow to know when the new question is related to the previous conversatio and when it isn't? But how to achieve this? Does anybody know how to get around this or can point to helpful articles to learn more about fixing this issue?
Thanks!