Truefoundry provides an easy way to build a QA system on your documents using RAG. While its easy to build a quick demo on Notebook using Langchain, taking a RAG system to production requires quite a lot of additional steps as follows:
- Infrastructure Complexity: RAG requires hosting a scaling query engine, highly available and fault tolerant Vector Database and a metadata store to keep track of the state of indexing. This requires deployment of multiple microservices and the setup to make them all work together.
- Incremental Indexing: Soon after launching a prototype, there is a need of being able to support addition, updation and deletion of documents in a incremental way.
- Access Control: There might be cases when the search answers need to vary based on the user making the query since the user might not have access to all documents.
- Guard Rails:
- Semantic Caching: Semantic Caching of queries can help improve user experience
Updated about 8 hours ago