A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces

TL;DR

A-RAG introduces a hierarchical retrieval interface for language models, enabling adaptive, multi-granularity information retrieval that improves open-domain question answering performance and scalability over existing retrieval-augmented generation methods.

Contribution

The paper presents A-RAG, a novel framework that allows language models to participate in retrieval decisions through hierarchical tools, enhancing flexibility and scalability.

Findings

A-RAG outperforms existing RAG methods on multiple QA benchmarks.

A-RAG effectively leverages model capabilities with fewer retrieved tokens.

A-RAG scales well with model size and compute resources.

Abstract

Frontier language models have demonstrated strong reasoning and long-horizon tool-use capabilities. However, existing RAG systems fail to leverage these capabilities. They still rely on two paradigms: (1) designing an algorithm that retrieves passages in a single shot and concatenates them into the model's input, or (2) predefining a workflow and prompting the model to execute it step-by-step. Neither paradigm allows the model to participate in retrieval decisions, preventing efficient scaling with model improvements. In this paper, we introduce A-RAG, an Agentic RAG framework that exposes hierarchical retrieval interfaces directly to the model. A-RAG provides three retrieval tools: keyword search, semantic search, and chunk read, enabling the agent to adaptively search and retrieve information across multiple granularities. Experiments on multiple open-domain QA benchmarks show that A-RAG consistently outperforms existing approaches with comparable or lower retrieved tokens, demonstrating that A-RAG effectively leverages model capabilities and dynamically adapts to different RAG tasks. We further systematically study how A-RAG scales with model size and test-time compute. We will release our code and evaluation suite to facilitate future research. Code and evaluation suite are available at https://github.com/Ayanami0730/arag.