CP-Router: An Uncertainty-Aware Router Between LLM and LRM

TL;DR

CP-Router is a model-agnostic, uncertainty-based routing framework that dynamically chooses between LLMs and LRMs for efficient and accurate question answering, reducing token usage while maintaining or improving performance.

Contribution

It introduces a training-free, uncertainty-guided routing method using conformal prediction and entropy-based criteria, enhancing LLM and LRM collaboration for diverse QA tasks.

Findings

Reduces token usage while maintaining accuracy in MCQA tasks.

Demonstrates robustness across diverse model pairings and open-ended QA.

Achieves improved efficiency and performance in multiple benchmarks.

Abstract

Recent advances in Large Reasoning Models (LRMs) have significantly improved long-chain reasoning capabilities over Large Language Models (LLMs). However, LRMs often produce unnecessarily lengthy outputs even for simple queries, leading to inefficiencies or even accuracy degradation compared to LLMs. To overcome this, we propose CP-Router, a training-free and model-agnostic routing framework that dynamically selects between an LLM and an LRM, demonstrated with multiple-choice question answering (MCQA) prompts. The routing decision is guided by the prediction uncertainty estimates derived via Conformal Prediction (CP), which provides rigorous coverage guarantees. To further refine the uncertainty differentiation across inputs, we introduce Full and Binary Entropy (FBE), a novel entropy-based criterion that adaptively selects the appropriate CP threshold. Experiments across diverse MCQA benchmarks, including mathematics, logical reasoning, and Chinese chemistry, demonstrate that CP-Router efficiently reduces token usage while maintaining or even improving accuracy compared to using LRM alone. We also extend CP-Router to diverse model pairings and open-ended QA, where it continues to demonstrate strong performance, validating its generality and robustness.