With the database transformed, we moved forward to the next step which was the application of an innovative concept known as RAG (Retrieval-Augmented Generation). It combines pre-trained AI language models like GPT-4 with information retrieved from an external knowledge base (like the legal document stored by the customer). It significantly improves conversational AI responses by retrieving information relevant to a specific context of questions asked by end-users. It was a great tool for our proof of concept but was ill-suited for the production phase as RAG has not yet reached the production-ready status. That’s when we switched to the OpenAI library.
To facilitate effective search functionalities within legal documents, we incorporated the Qdrant vector database, designed to bolster semantic search and guarantee accurate and contextually aware information retrieval. For the secure storage and management of user interactions and conversation history, we opted for PostgreSQL as our chosen database solution

The foundation of our system rested on embedding legal documents, and for this task, we utilized the text-embedding-ada002 model provided by OpenAI. Additionally, we integrated the GPT-3.5 and GPT-4 models as our primary Language Models (LLM) to generate responses and engage in effective legal discourse. In upholding a secure and compliant setting, our team deployed OpenAI's moderation endpoint to guarantee adherence to OpenAI's policies. Prioritizing data privacy and anonymization, we employed AWS cloud services to protect user information and interactions. As a result, we designed and implemented a custom AI chat assistant able to answer legal questions in a contextual, swift, and secure manner.