Not All Queries Need Deep Thought: CoFiCot for Adaptive Coarse-to-fine Stateful Refinement

TL;DR

CoFiCot introduces an adaptive coarse-to-fine framework for large language model inference, dynamically allocating computational resources based on query difficulty to improve reasoning accuracy and efficiency.

Contribution

It presents a novel multi-metric classifier and a stateful correction process that adaptively refines model outputs, addressing the uniform computation paradox in test-time inference.

Findings

Improves reasoning accuracy on complex queries

Reduces unnecessary computation on simple queries

Enhances logical coherence with stateful correction

Abstract

Scaling test-time computation enhances LLM reasoning ability but faces a uniform computation paradox. Allocating identical resources leads to over-correction on simple tasks and insufficient refinement on complex ones. To address this, we propose CoFiCot, a coarse-to-fine adaptive framework that dynamically tailors inference strategies to problem difficulty. Specifically, we implement a multi-metric classifier that triages queries by synthesizing semantic entropy, consensus reliability, and predicted reasoning depth . This enables a differentiated refinement stage that applies efficient aggregation for simple queries while routing complex ones to a context-aware correction loop . We formalize correction as a stateful sequential propagation process , where each repair is strictly conditioned on the verified history of prior rectifications. By integrating Process Reward Models (PRMs) within this state-dependent trajectory, CoFiCot effectively bridges the gap between granular error localization and global logical coherence, preventing the context fragmentation typical of stateless refinement methods.