SkillOrchestra: Learning to Route Agents via Skill Transfer

TL;DR

SkillOrchestra introduces a skill-aware orchestration framework that learns fine-grained skills and models agent competence to improve routing efficiency and effectiveness in compound AI systems, outperforming RL-based methods.

Contribution

It proposes a novel skill-based approach for agent routing that reduces training costs and improves scalability and interpretability over existing RL-based orchestrators.

Findings

Outperforms state-of-the-art RL-based orchestrators by up to 22.5%.

Reduces learning costs by 700x and 300x compared to existing methods.

Demonstrates effectiveness across ten benchmark tasks.

Abstract

Compound AI systems promise capabilities beyond those of individual models, yet their success depends critically on effective orchestration. Existing routing approaches face two limitations: (1) input-level routers make coarse query-level decisions that ignore evolving task requirements; (2) RL-trained orchestrators are expensive to adapt and often suffer from routing collapse, repeatedly invoking one strong but costly option in multi-turn scenarios. We introduce SkillOrchestra, a framework for skill-aware orchestration. Instead of directly learning a routing policy end-to-end, SkillOrchestra learns fine-grained skills from execution experience and models agent-specific competence and cost under those skills. At deployment, the orchestrator infers the skill demands of the current interaction and selects agents that best satisfy them under an explicit performance-cost trade-off. Extensive experiments across ten benchmarks demonstrate that SkillOrchestra outperforms SoTA RL-based orchestrators by up to 22.5% with 700x and 300x learning cost reduction compared to Router-R1 and ToolOrchestra, respectively. These results show that explicit skill modeling enables scalable, interpretable, and sample-efficient orchestration, offering a principled alternative to data-intensive RL-based approaches. The code is available at: https://github.com/jiayuww/SkillOrchestra.