RAGPulse: An Open-Source RAG Workload Trace to Optimize RAG Serving Systems

TL;DR

RAGPulse provides a detailed, real-world workload trace dataset for Retrieval-Augmented Generation systems, enabling researchers to optimize RAG service performance through realistic data analysis and validation.

Contribution

This paper introduces RAGPulse, the first open-source RAG workload trace dataset capturing real-world dynamics for performance optimization research.

Findings

Real-world RAG workloads show temporal locality.

Workloads exhibit highly skewed hot document access patterns.

Dataset enables development of content-aware optimization strategies.

Abstract

Retrieval-Augmented Generation (RAG) is a critical paradigm for building reliable, knowledge-intensive Large Language Model (LLM) applications. However, the multi-stage pipeline (retrieve, generate) and unique workload characteristics (e.g., knowledge dependency) of RAG systems pose significant challenges for serving performance optimization. Existing generic LLM inference traces fail to capture these RAG-specific dynamics, creating a significant performance gap between academic research and real-world deployment. To bridge this gap, this paper introduces RAGPulse, an open-source RAG workload trace dataset. This dataset was collected from an university-wide Q&A system serving that has served more than 40,000 students and faculties since April 2024. We detail RAGPulse's system architecture, its privacy-preserving hash-based data format, and provide an in-depth statistical analysis. Our analysis reveals that real-world RAG workloads exhibit significant temporal locality and a highly skewed hot document access pattern. RAGPulse provides a high-fidelity foundation for researchers to develop and validate novel optimization strategies for RAG systems, such as content-aware batching and retrieval caching, ultimately enhancing the efficiency and reliability of RAG services. The code is available at https://github.com/flashserve/RAGPulse.