Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models

TL;DR

This paper investigates how large language models' internal representations evolve during reasoning tasks, revealing that training influences the distributional shifts in internal states rather than static representations, and that reasoning involves dynamic state transitions.

Contribution

It introduces a representational analysis framework to study reasoning evolution in large language models, highlighting the importance of internal state transitions over static representations during reasoning.

Findings

Post-training yields limited static representation improvements.

Reasoning involves significant distributional shifts in internal states.

Final representations strongly correlate with correct outputs.

Abstract

Large Language Models have achieved remarkable performance on reasoning tasks, motivating research into how this ability evolves during training. Prior work has primarily analyzed this evolution via explicit generation outcomes, treating the reasoning process as a black box and obscuring internal changes. To address this opacity, we introduce a representational perspective to investigate the dynamics of the model's internal states. Through comprehensive experiments across models at various training stages, we discover that post-training yields only limited improvement in static initial representation quality. Furthermore, we reveal that, distinct from non-reasoning tasks, reasoning involves a significant continuous distributional shift in representations during generation. Comparative analysis indicates that post-training empowers models to drive this transition toward a better distribution for task solving. To clarify the relationship between internal states and external outputs, statistical analysis confirms a high correlation between generation correctness and the final representations; while counterfactual experiments identify the semantics of the generated tokens, rather than additional computation during inference or intrinsic parameter differences, as the dominant driver of the transition. Collectively, we offer a novel understanding of the reasoning process and the effect of training on reasoning enhancement, providing valuable insights for future model analysis and optimization.