RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routing

TL;DR

This paper introduces RAGRouter-Bench, a comprehensive dataset and benchmark for evaluating adaptive retrieval-augmented generation routing, emphasizing context-aware paradigm selection to optimize effectiveness and efficiency.

Contribution

It presents the first systematic benchmark for adaptive RAG routing, integrating diverse query types, corpus indicators, and evaluation protocols for comprehensive analysis.

Findings

No single RAG paradigm is optimal for all query-corpus pairs.

Adaptive routing improves effectiveness-efficiency trade-offs over fixed paradigms.

Query-corpus compatibility is crucial for effective RAG system design.

Abstract

Retrieval-augmented generation (RAG) has evolved into a family of paradigms with distinct performance profiles and resource demands, turning paradigm selection into a multi-criteria, context-dependent decision problem. Nevertheless, existing studies largely focus on isolated method improvements or query-only benchmarking, without systematically examining how RAG paradigms behave across diverse query-corpus contexts and effectiveness-efficiency trade-offs. In this work, we introduce RAGRouter-Bench, the first dataset and benchmark for adaptive RAG routing. Grounded in query-corpus compatibility, the benchmark integrates three canonical query types, fine-grained corpus indicators capturing structural and semantic properties, and a unified protocol for evaluating both generation quality and resource consumption. Then, we implement standardized RAG paradigms with multiple backbone LLMs across all query-corpus combinations, constructing a comprehensive benchmark with quantitative metrics and LLM-as-a-Judge evaluations to inform context-aware and cost-effective RAG routing decisions. We further formulate routing as context-dependent paradigm selection and benchmark a range of query-corpus routers on the constructed dataset. Extensive experiments demonstrate that no one-size-fits-all paradigm exists across query-corpus pairs, and that adaptive routing yields more favorable effectiveness-efficiency trade-offs than fixed paradigm selection. These findings establish query-corpus compatibility as a central principle for adaptive RAG routing and position RAGRouter-Bench as a systematic testbed for next-generation RAG systems.