Statistical Early Stopping for Reasoning Models

TL;DR

This paper proposes statistically grounded early stopping methods for reasoning models that monitor uncertainty signals during generation, aiming to improve efficiency and reliability, especially in math reasoning tasks.

Contribution

It introduces two novel early stopping techniques based on uncertainty signals, with theoretical guarantees and empirical validation across multiple reasoning tasks and models.

Findings

Uncertainty-aware early stopping improves reasoning efficiency.

The methods enhance reliability in LLM reasoning.

Significant gains observed in math reasoning tasks.

Abstract

While LLMs have seen substantial improvement in reasoning capabilities, they also sometimes overthink, generating unnecessary reasoning steps, particularly under uncertainty, given ill-posed or ambiguous queries. We introduce statistically principled early stopping methods that monitor uncertainty signals during generation to mitigate this issue. Our first approach is parametric: it models inter-arrival times of uncertainty keywords as a renewal process and applies sequential testing for stopping. Our second approach is nonparametric and provides finite-sample guarantees on the probability of halting too early on well-posed queries. We conduct empirical evaluations on reasoning tasks across several domains and models. Our results indicate that uncertainty-aware early stopping can improve both efficiency and reliability in LLM reasoning, and we observe especially significant gains for math reasoning.