SkillRouter: Skill Routing for LLM Agents at Scale

TL;DR

SkillRouter is a scalable, efficient skill routing system for large LLM agent skill repositories, significantly improving accuracy by leveraging full skill text rather than just metadata.

Contribution

We introduce SkillRouter, a compact retrieve-and-rerank pipeline that enhances skill routing accuracy and efficiency in large, overlapping skill ecosystems.

Findings

Hiding skill bodies reduces routing accuracy by 31-44 percentage points.

SkillRouter achieves 74.0% Hit@1, outperforming baselines in accuracy.

Routing improvements lead to better task success across multiple agents.

Abstract

Reusable skills let LLM agents package task-specific procedures, tool affordances, and execution guidance into modular building blocks. As skill ecosystems grow to tens of thousands of entries, exposing every skill at inference time becomes infeasible. This creates a skill-routing problem: given a user task, the system must identify relevant skills before downstream planning or execution. Existing agent stacks often rely on progressive disclosure, exposing only skill names and descriptions while hiding the full implementation body. We examine this design choice on a SkillsBench-derived benchmark with approximately 80K candidate skills, targeting the practically important setting of large skill registries with heavy overlap. Across representative sparse, dense, and reranking baselines on this setting, hiding the skill body causes a 31--44 percentage point drop in routing accuracy, showing that full skill text is a critical routing signal in this setting rather than a minor metadata refinement. Motivated by this finding, we present SkillRouter, a compact 1.2B full-text retrieve-and-rerank pipeline. SkillRouter achieves 74.0% Hit@1 on our benchmark -- the strongest average top-1 routing performance among the baselines we evaluate -- while using 13$\times$ fewer parameters and running 5.8$\times$ faster than the strongest base pipeline. The ranking gains further generalize to a supplementary benchmark independently constructed from three skill sources. In a complementary end-to-end study across four coding agents, routing gains transfer to improved task success, with larger gains for more capable agents.