CoRefine: Confidence-Guided Self-Refinement for Adaptive Test-Time Compute

TL;DR

CoRefine is a confidence-guided self-refinement method for large language models that reduces compute by selectively halting or re-examining based on confidence, achieving high accuracy with fewer tokens.

Contribution

It introduces a lightweight controller that guides self-refinement in LLMs, enabling efficient reasoning with significantly less compute and high confidence accuracy.

Findings

Achieves 92.6% confidence halting precision

Reduces token usage by roughly 190-fold compared to baseline

Enables scalable reasoning with a modular primitive

Abstract

Large Language Models (LLMs) often rely on test-time scaling via parallel decoding (for example, 512 samples) to boost reasoning accuracy, but this incurs substantial compute. We introduce CoRefine, a confidence-guided self-refinement method that achieves competitive accuracy using a fraction of the tokens via a lightweight 211k-parameter Conv1D controller atop a frozen LLM. The controller consumes full-trace confidence to decide whether to halt, re-examine, or try a different approach, enabling targeted self-correction with an average of 2.7 refinement steps per problem and roughly 190-fold token reduction relative to 512-sample baselines. Across diverse reasoning benchmarks and three open-source models, the controller achieves 92.6 percent precision when it confidently halts, indicating that confidence dynamics reliably signal correctness without ground-truth verification. We extend this to CoRefine-Tree, a hybrid sequential-parallel variant that adaptively balances exploration and exploitation, with easy serving integration and verifier compatibility. By treating confidence as a control signal rather than a correctness guarantee, CoRefine provides a modular primitive for scalable reasoning and agentic settings with imperfect verifiers.