Build QA using RAG

Truefoundry provides an easy way to build a QA system on your documents using RAG. While its easy to build a quick demo on Notebook using Langchain, taking a RAG system to production requires quite a lot of additional steps as follows:

1. Infrastructure Complexity: RAG requires hosting a scaling query engine, highly available and fault tolerant Vector Database and a metadata store to keep track of the state of indexing. This requires deployment of multiple microservices and the setup to make them all work together.
2. Incremental Indexing: Soon after launching a prototype, there is a need of being able to support addition, updation and deletion of documents in a incremental way.
3. Access Control: There might be cases when the search answers need to vary based on the user making the query since the user might not have access to all documents.
4. Guard Rails:
5. Evaluation:
6. Semantic Caching: Semantic Caching of queries can help improve user experience