TagRouter: Learning Route to LLMs through Tags for Open-Domain Text Generation Tasks

TL;DR

TagRouter is a training-free model routing method that enhances large language model ensemble efficiency for open-domain text generation, outperforming baselines in accept rate and cost reduction.

Contribution

It introduces a scalable, training-free routing approach that improves model ensemble performance and cost-efficiency in large-scale LLM applications.

Findings

Outperforms 13 baseline methods in experiments

Increases system accept rate by 6.15%

Reduces costs by 17.20%

Abstract

Model routing allocates queries to the suitable model, improving system performance while reducing costs. However, existing routing methods face practical limitations that hinder scalability in large-scale applications and struggle to keep up with the rapid growth of the large language model (LLM) ecosystem. To tackle these challenges, we propose TagRouter, a training-free model routing method designed to optimize the synergy among multiple LLMs for open-domain text generation tasks. Experimental results demonstrate that TagRouter outperforms 13 baseline methods, increasing the accept rate of system by 6.15% and reducing costs by 17.20%, achieving optimal cost-efficiency. Our findings provides the LLM community with an efficient and scalable solution for model ensembling, offering users an evolvable "super model."