Reasoning Models Don't Just Think Longer, They Move Differently

TL;DR

This paper investigates whether reasoning-trained language models follow different internal trajectories when solving harder problems, revealing domain-dependent differences in their reasoning processes after adjusting for generation length.

Contribution

It introduces a length correction method for trajectory analysis and demonstrates that reasoning training influences internal model dynamics in a domain-specific manner.

Findings

Corrected trajectory geometry correlates with problem difficulty across domains.

Reasoning-trained models show more direct and less heterogeneous trajectories in code tasks.

Length correction is essential for meaningful trajectory analysis.

Abstract

Reasoning-trained language models often spend more tokens on harder problems, but longer chains of thought do not show whether a model is merely computing for more steps or following a different internal trajectory. We study this distinction through hidden-state trajectories during chain-of-thought generation across competitive programming, mathematics, and Boolean satisfiability. Raw trajectory geometry is strongly shaped by generation length: longer generations mechanically alter path statistics, so difficulty-dependent comparisons are misleading without adjustment. After residualizing trajectory statistics on length, difficulty remains systematically coupled to corrected trajectory geometry across all domains studied. The clearest reasoning-specific separation appears in the code domain, where harder problems show more direct corrected trajectories and less heterogeneous local curvature in reasoning-trained models than in matched instruction-tuned baselines. Corrected difficulty-geometry coupling is weaker, but still present, in mathematics and Boolean satisfiability. Prompt-stage linear probes do not mirror the code-domain separation, and behavioral annotations show that stronger corrected coupling co-occurs with strategy shifts and uncertainty monitoring. Together, these findings establish length correction as a prerequisite for generation-time trajectory analysis and show that reasoning training can be associated with distinct corrected trajectory geometry, with the strength of the effect depending on the domain.