Orchestrating Intelligence: Confidence-Aware Routing for Efficient Multi-Agent Collaboration across Multi-Scale Models

TL;DR

This paper introduces OI-MAS, a confidence-aware multi-agent framework that adaptively routes tasks across heterogeneous models, significantly improving efficiency and accuracy in multi-agent reasoning tasks.

Contribution

The paper presents a novel adaptive routing and model-selection mechanism for multi-agent systems using multi-scale LLMs, enhancing efficiency and performance.

Findings

Achieves up to 12.88% accuracy improvement

Reduces computational cost by up to 79.78%

Demonstrates effective dynamic model selection

Abstract

While multi-agent systems (MAS) have demonstrated superior performance over single-agent approaches in complex reasoning tasks, they often suffer from significant computational inefficiencies. Existing frameworks typically deploy large language models (LLMs) uniformly across all agent roles, failing to account for the varying cognitive demands of different reasoning stages. We address this inefficiency by proposing OI-MAS framework, a novel multi-agent framework that implements an adaptive model-selection policy across a heterogeneous pool of multi-scale LLMs. Specifically, OI-MAS introduces a state-dependent routing mechanism that dynamically selects agent roles and model scales throughout the reasoning process. In addition, we introduce a confidence-aware mechanism that selects appropriate model scales conditioned on task complexity, thus reducing unnecessary reliance on large-scale models. Experimental results show that OI-MAS consistently outperforms baseline multi-agent systems, improving accuracy by up to 12.88\% while reducing cost by up to 79.78\%.