RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing
Ruihan Jin, Pengpeng Shao, Zhengqi Wen, Jinyang Wu, Mingkuan Feng, Shuai Zhang, Jianhua Tao

TL;DR
RadialRouter introduces a novel Transformer-based framework with a radial structure for efficient and robust large language model routing, significantly improving selection accuracy and adaptability over existing methods.
Contribution
It proposes RadialRouter, a new LLM routing framework utilizing RadialFormer and a combined loss function to better capture query-LLM relationships and enhance robustness.
Findings
Outperforms existing routing methods by 9.2% and 5.8% in key scenarios.
Demonstrates high adaptability to different performance-cost trade-offs.
Shows practical potential with dynamic LLM pool management.
Abstract
The rapid advancements in large language models (LLMs) have led to the emergence of routing techniques, which aim to efficiently select the optimal LLM from diverse candidates to tackle specific tasks, optimizing performance while reducing costs. Current LLM routing methods are limited in effectiveness due to insufficient exploration of the intrinsic connection between user queries and the characteristics of LLMs. To address this issue, in this paper, we present RadialRouter, a novel framework for LLM routing which employs a lightweight Transformer-based backbone with a radial structure named RadialFormer to articulate the query-LLMs relationship. The optimal LLM selection is performed based on the final states of RadialFormer. The pipeline is further refined by an objective function that combines Kullback-Leibler divergence with the query-query contrastive loss to enhance robustness.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Complex Network Analysis Techniques · Advanced Graph Neural Networks
