DiSRouter: Distributed Self-Routing for LLM Selections

TL;DR

DiSRouter introduces a distributed, self-aware routing system for LLMs, enabling flexible, scalable, and effective query routing by leveraging each model's self-assessment capabilities, outperforming traditional centralized methods.

Contribution

This work proposes a novel distributed self-routing paradigm for LLMs, utilizing self-awareness training to improve query routing without relying on external routers.

Findings

Outperforms existing routing methods in utility.

Effectively distinguishes easy and hard queries.

Generalizes well to out-of-domain tasks.

Abstract

The proliferation of Large Language Models (LLMs) has created a diverse ecosystem of models with highly varying performance and costs, necessitating effective query routing to balance performance and expense. Current routing systems often rely on a centralized external router trained on a fixed set of LLMs, making them inflexible and prone to poor performance since the small router can not fully understand the knowledge boundaries of different LLMs. We introduce DiSRouter (Distributed Self-Router), a novel paradigm that shifts from centralized control to distributed routing. In DiSRouter, a query traverses a network of LLM agents, each independently deciding whether to answer or route to other agents based on its own self-awareness, its ability to judge its competence. This distributed design offers superior flexibility, scalability, and generalizability. To enable this, we propose a two-stage Self-Awareness Training pipeline that enhances each LLM's self-awareness. Extensive experiments demonstrate that DiSRouter significantly outperforms existing routing methods in utility across various scenarios, effectively distinguishes between easy and hard queries, and shows strong generalization to out-of-domain tasks. Our work validates that leveraging an LLM's intrinsic self-awareness is more effective than external assessment, paving the way for more modular and efficient multi-agent systems.