RAGOps: Operating and Managing Retrieval-Augmented Generation Pipelines
Xiwei Xu, Hans Weytjens, Dawen Zhang, Qinghua Lu, Ingo Weber, Liming Zhu

TL;DR
This paper introduces RAGOps, a framework for managing retrieval-augmented generation systems by addressing data management, lifecycle, and quality challenges to improve enterprise LLM applications.
Contribution
It characterizes RAG architecture, outlines management lifecycle, and defines design considerations for operationalizing RAG systems in production environments.
Findings
Characterized RAG architecture using 4+1 model view
Outlined RAG system lifecycle integrating data and model management
Presented two RAG application use cases with operational considerations
Abstract
Recent studies show that 60% of LLM-based compound systems in enterprise environments leverage some form of retrieval-augmented generation (RAG), which enhances the relevance and accuracy of LLM (or other genAI) outputs by retrieving relevant information from external data sources. LLMOps involves the practices and techniques for managing the lifecycle and operations of LLM compound systems in production environments. It supports enhancing LLM systems through continuous operations and feedback evaluation. RAGOps extends LLMOps by incorporating a strong focus on data management to address the continuous changes in external data sources. This necessitates automated methods for evaluating and testing data operations, enhancing retrieval relevance and generation quality. In this paper, we (1) characterize the generic architecture of RAG applications based on the 4+1 model view for…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInformation Retrieval and Search Behavior · Advanced Database Systems and Queries · Software Engineering Research
