FlyRoute: Self-Evolving Agent Profiling via Data Flywheel for Adaptive Task Routing

TL;DR

FlyRoute is a self-evolving profiling framework that enhances adaptive task routing by continuously growing and distilling agent capability evidence from real traffic, significantly improving routing accuracy.

Contribution

The paper introduces FlyRoute, a novel data-efficient, self-evolving profiling system that dynamically updates agent profiles and improves routing accuracy in enterprise settings.

Findings

FlyRoute improves zero-shot routing accuracy from 72.57% to 78.04%.

After streaming 7,211 queries, accuracy reaches 89.83%.

Consistent gains across four expert domains.

Abstract

Enterprise routers assign queries to expert agents, yet deployed profiles stay static while agents evolve (prompts, tools, models), and developers rarely keep descriptions or exemplars current. We present FlyRoute, a self-evolving profiling framework that grows capability evidence from real traffic: dispatch candidates, quality-gate successful pairs into each agent's success store, periodically distill evidence into learned capability descriptions, and inject those descriptions together with BM25-retrieved successes into an LLM router. To make this flywheel data-efficient, FlyRoute introduces a targeted exploration policy that combines profile uncertainty, BM25 relevance, and lexical novelty, prioritizing under-profiled agents only for plausible queries and avoiding redundant evidence collection. In experiments on our proprietary enterprise developer-support dataset of real routed queries, FlyRoute improves a same-backbone zero-shot LLM router from 72.57% to 78.04% with only five seed queries per agent, showing that profile retrieval already strengthens cold-start routing. After streaming 7,211 labeled training queries through the flywheel, accuracy rises to 89.83% (+17.26pp over zero-shot; +11.79pp over cold start), with consistent gains across four expert domains under standard routing accuracy on single-gold test queries.