Lightweight Query Routing for Adaptive RAG: A Baseline Study on RAGRouter-Bench

TL;DR

This study evaluates lightweight classifier-based routing strategies for Retrieval-Augmented Generation, demonstrating high accuracy and significant token savings on the RAGRouter-Bench benchmark across various domains.

Contribution

First systematic evaluation of lightweight classifiers for query routing in RAG, establishing a strong baseline and analyzing feature effectiveness and domain-specific challenges.

Findings

TF-IDF with SVM achieves 93.2% accuracy and 28.1% token savings.

Lexical features outperform semantic embeddings in routing accuracy.

Medical queries are more challenging to route than legal queries.

Abstract

Retrieval-Augmented Generation pipelines span a wide range of retrieval strategies that differ substantially in token cost and capability. Selecting the right strategy per query is a practical efficiency problem, yet no routing classifiers have been trained on RAGRouter-Bench \citep{wang2026ragrouterbench}, a recently released benchmark of $7,727$ queries spanning four knowledge domains, each annotated with one of three canonical query types: factual, reasoning, and summarization. We present the first systematic evaluation of lightweight classifier-based routing on this benchmark. Five classical classifiers are evaluated under three feature regimes, namely, TF-IDF, MiniLM sentence embeddings \citep{reimers2019sbert}, and hand-crafted structural features, yielding 15 classifier feature combinations. Our best configuration, TF-IDF with an SVM, achieves a macro-averaged F1 of $\mathbf{0.928}$ and an accuracy of $\mathbf{93.2\%}$, while simulating $\mathbf{28.1\%}$ token savings relative to always using the most expensive paradigm. Lexical TF-IDF features outperform semantic sentence embeddings by $3.1$ macro-F1 points, suggesting that surface keyword patterns are strong predictors of query-type complexity. Domain-level analysis reveals that medical queries are hardest to route and legal queries most tractable. These results establish a reproducible query-side baseline and highlight the gap that corpus-aware routing must close.