Oops, Wait: Token-Level Signals as a Lens into LLM Reasoning

TL;DR

This paper investigates how token-level signals like "wait" and "therefore" in large language models relate to reasoning, revealing their correlation with correctness and how training strategies influence their use.

Contribution

It systematically analyzes token probabilities across models and training strategies, providing insights into the role of token signals in LLM reasoning.

Findings

Specific tokens correlate with reasoning correctness

Training strategies influence token signal usage

Models fine-tuned on small datasets partially exploit token signals

Abstract

The emergence of discourse-like tokens such as "wait" and "therefore" in large language models (LLMs) has offered a unique window into their reasoning processes. However, systematic analyses of how such signals vary across training strategies and model scales remain lacking. In this paper, we analyze token-level signals through token probabilities across various models. We find that specific tokens strongly correlate with reasoning correctness, varying with training strategies while remaining stable across model scales. A closer look at the "wait" token in relation to answer probability demonstrates that models fine-tuned on small-scale datasets acquire reasoning ability through such signals but exploit them only partially. This work provides a systematic lens to observe and understand the dynamics of LLM reasoning.