Cache Mechanism for Agent RAG Systems

TL;DR

This paper introduces ARC, a caching framework for RAG systems that dynamically manages compact, high-relevance knowledge bases, significantly reducing storage and latency while improving answer accuracy in LLM agents.

Contribution

ARC is a novel, annotation-free cache management method that adapts to query patterns and embedding geometry, enhancing RAG system efficiency and effectiveness.

Findings

Reduces storage to 0.015% of original corpus

Increases has-answer rate up to 79.8%

Decreases retrieval latency by 80%

Abstract

Recent advances in Large Language Model (LLM)-based agents have been propelled by Retrieval-Augmented Generation (RAG), which grants the models access to vast external knowledge bases. Despite RAG's success in improving agent performance, agent-level cache management, particularly constructing, maintaining, and updating a compact, relevant corpus dynamically tailored to each agent's need, remains underexplored. Therefore, we introduce ARC (Agent RAG Cache Mechanism), a novel, annotation-free caching framework that dynamically manages small, high-value corpora for each agent. By synthesizing historical query distribution patterns with the intrinsic geometry of cached items in the embedding space, ARC automatically maintains a high-relevance cache. With comprehensive experiments on three retrieval datasets, our experimental results demonstrate that ARC reduces storage requirements to 0.015% of the original corpus while offering up to 79.8% has-answer rate and reducing average retrieval latency by 80%. Our results demonstrate that ARC can drastically enhance efficiency and effectiveness in RAG-powered LLM agents.