Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarios

TL;DR

This paper introduces TRouter, a task-aware LLM routing system that uses multi-level task-profile-guided data synthesis to improve performance in cold-start scenarios by modeling query-specific cost and accuracy.

Contribution

It presents a novel hierarchical task taxonomy and a task-type-aware routing approach that enhances LLM routing in cold-start and in-domain settings.

Findings

TRouter improves routing effectiveness in cold-start scenarios.

The data synthesis framework alleviates cold-start issues.

TRouter outperforms existing routing methods across multiple benchmarks.

Abstract

Large language models (LLMs) exhibit substantial variability in performance and computational cost across tasks and queries, motivating routing systems that select models to meet user-specific cost-performance trade-offs. However, existing routers generalize poorly in cold-start scenarios where in-domain training data is unavailable. We address this limitation with a multi-level task-profile-guided data synthesis framework that constructs a hierarchical task taxonomy and produces diverse question-answer pairs to approximate the test-time query distribution. Building on this, we introduce TRouter, a task-type-aware router approach that models query-conditioned cost and performance via latent task-type variables, with prior regularization derived from the synthesized task taxonomy. This design enhances TRouter's routing utility under both cold-start and in-domain settings. Across multiple benchmarks, we show that our synthesis framework alleviates cold-start issues and that TRouter delivers effective LLM routing.