Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning

TL;DR

STEER is a confidence-guided, stepwise routing framework that dynamically allocates reasoning tasks between smaller and larger LLMs, significantly reducing inference costs while maintaining or improving accuracy across diverse benchmarks.

Contribution

The paper introduces STEER, a novel domain-agnostic, internal-confidence-based routing method that eliminates the need for external models or costly data synthesis for cost-efficient LLM reasoning.

Findings

Achieves up to 20% accuracy improvement with 48% less FLOPs.

Outperforms baselines relying on trained external modules.

Demonstrates robustness across multiple domains and benchmarks.

Abstract

Recent advances in Large Language Models (LLMs) - particularly model scaling and test-time techniques - have greatly enhanced the reasoning capabilities of language models at the expense of higher inference costs. To lower inference costs, prior works train router models or deferral mechanisms that allocate easy queries to a small, efficient model, while forwarding harder queries to larger, more expensive models. However, these trained router models often lack robustness under domain shifts and require expensive data synthesis techniques such as Monte Carlo rollouts to obtain sufficient ground-truth routing labels for training. In this work, we propose Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning (STEER), a domain-agnostic framework that performs fine-grained, step-level routing between smaller and larger LLMs without utilizing external models. STEER leverages confidence scores from the smaller model's logits prior to generating a reasoning step, so that the large model is invoked only when necessary. Extensive evaluations using different LLMs on a diverse set of challenging benchmarks across multiple domains such as Mathematical Reasoning, Multi-Hop QA, and Planning tasks indicate that STEER achieves competitive or enhanced accuracy while reducing inference costs (up to +20% accuracy with 48% less FLOPs compared to solely using the larger model on AIME), outperforming baselines that rely on trained external modules. Our results establish model-internal confidence as a robust, domain-agnostic signal for model routing, offering a scalable pathway for efficient LLM deployment.