Tracing Uncertainty in Language Model "Reasoning"

TL;DR

This paper investigates how uncertainty quantification can reveal the dynamics of language model reasoning, enabling early detection of correct or incorrect outputs through uncertainty profile analysis.

Contribution

It introduces a novel uncertainty trace profile method that predicts answer correctness with high accuracy across multiple language models.

Findings

Uncertainty profiles predict correctness with AUROC up to 0.807.

Early tokens contain enough information to predict correctness with AUROC 0.801.

Correct traces show a steeper, less linear decline in uncertainty.

Abstract

Language model (LM) "reasoning", commonly described as Chain-of-Thought or test-time scaling, often improves benchmark performance, but the dynamics underlying this process remain poorly understood. We study these dynamics through the lens of uncertainty quantification by treating the "reasoning" traces, the intermediate token sequences generated by LMs, as evolving model states. We summarize each trace by an uncertainty trace profile: a small set of features describing the shape of the uncertainty signal over its trace, such as its slope and linearity. We find that across five LMs evaluated on GSM8K and ProntoQA, these profiles predict whether a trace yields a correct final answer with AUROC up to 0.807, improving markedly on recent related work. We reach AUROC 0.801 using only the first few hundred tokens of full traces, suggesting that errors can be detected early in the generation. A detailed comparison of correct and incorrect traces further reveals qualitatively distinct uncertainty profiles, with correct traces showing a steeper and less linear decline in uncertainty. Together, the results suggest that our method, grounded in decision-making under uncertainty, provides a principled lens for studying the generative process underlying LM "reasoning".