RadialRouter: Structured Representation for Efficient and Robust Large Language Models Routing

TL;DR

RadialRouter introduces a novel Transformer-based framework with a radial structure for efficient and robust large language model routing, significantly improving selection accuracy and adaptability over existing methods.

Contribution

It proposes RadialRouter, a new LLM routing framework utilizing RadialFormer and a combined loss function to better capture query-LLM relationships and enhance robustness.

Findings

Outperforms existing routing methods by 9.2% and 5.8% in key scenarios.

Demonstrates high adaptability to different performance-cost trade-offs.

Shows practical potential with dynamic LLM pool management.

Abstract

The rapid advancements in large language models (LLMs) have led to the emergence of routing techniques, which aim to efficiently select the optimal LLM from diverse candidates to tackle specific tasks, optimizing performance while reducing costs. Current LLM routing methods are limited in effectiveness due to insufficient exploration of the intrinsic connection between user queries and the characteristics of LLMs. To address this issue, in this paper, we present RadialRouter, a novel framework for LLM routing which employs a lightweight Transformer-based backbone with a radial structure named RadialFormer to articulate the query-LLMs relationship. The optimal LLM selection is performed based on the final states of RadialFormer. The pipeline is further refined by an objective function that combines Kullback-Leibler divergence with the query-query contrastive loss to enhance robustness. Experimental results on RouterBench show that RadialRouter significantly outperforms existing routing methods by 9.2\% and 5.8\% in the Balance and Cost First scenarios, respectively. Additionally, its adaptability toward different performance-cost trade-offs and the dynamic LLM pool demonstrates practical application potential.