One-Token Verification for Reasoning Correctness Estimation

TL;DR

This paper introduces One-Token Verification, a method enabling LLMs to estimate reasoning correctness during generation with a single forward pass, improving efficiency and reliability in complex reasoning tasks.

Contribution

The paper presents a novel, learnable token-based verification method integrated into LLMs that accurately assesses reasoning correctness and reduces inference costs.

Findings

OTV outperforms existing verifiers on mathematical reasoning benchmarks.

OTV enables up to 90% reduction in token usage through early stopping.

OTV supports token-level correctness estimation without disrupting primary reasoning.

Abstract

Recent breakthroughs in large language models (LLMs) have led to notable successes in complex reasoning tasks, such as mathematical problem solving. A common strategy for improving performance is parallel thinking, in which multiple reasoning traces are generated and the final prediction is made using aggregation schemes like majority voting or best-of-$N$ decoding. However, two key challenges persist. First, multi-sample decoding incurs substantial inference latency, especially for long-form outputs. Second, effective mechanisms for reliably assessing the correctness of individual reasoning traces are still limited. To address these challenges, we introduce One-Token Verification (OTV), a computational method that estimates reasoning correctness in a single forward pass during generation. OTV is activated by a learnable token and integrated into the LLM via low-rank adaptation to probe internal reasoning signals through the key-value cache, supporting token-level correctness estimation at any stage of generation without disrupting primary reasoning. Experiments on mathematical reasoning benchmarks demonstrate that OTV consistently surpasses existing verifiers. Additionally, OTV reduces token usage by up to $90\%$ through correctness-guided early termination, prioritizing shorter, more reliable solutions.