RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models

TL;DR

RAGRouter introduces a dynamic routing method for multiple retrieval-augmented language models, improving task performance by considering document influence and knowledge shifts, outperforming existing static routing approaches.

Contribution

It formally defines the retrieval-augmented LLM routing problem and proposes RAGRouter, a novel contrastive learning-based routing framework that adapts to document influence.

Findings

RAGRouter outperforms individual LLMs and existing routing methods.

It achieves a strong performance-efficiency trade-off under low-latency constraints.

Extensive experiments validate its effectiveness across diverse tasks and models.

Abstract

Retrieval-Augmented Generation (RAG) significantly improves the performance of Large Language Models (LLMs) on knowledge-intensive tasks. However, varying response quality across LLMs under RAG necessitates intelligent routing mechanisms, which select the most suitable model for each query from multiple retrieval-augmented LLMs via a dedicated router model. We observe that external documents dynamically affect LLMs' ability to answer queries, while existing routing methods, which rely on static parametric knowledge representations, exhibit suboptimal performance in RAG scenarios. To address this, we formally define the new retrieval-augmented LLM routing problem, incorporating the influence of retrieved documents into the routing framework. We propose RAGRouter, a RAG-aware routing design, which leverages document embeddings and RAG capability embeddings with contrastive learning to capture knowledge representation shifts and enable informed routing decisions. Extensive experiments on diverse knowledge-intensive tasks and retrieval settings, covering open and closed-source LLMs, show that RAGRouter outperforms the best individual LLM and existing routing methods. With an extended score-threshold-based mechanism, it also achieves strong performance-efficiency trade-offs under low-latency constraints. The code and data are available at https://github.com/OwwO99/RAGRouter.