LLM Reasoning Is Latent, Not the Chain of Thought

TL;DR

This paper advocates for studying LLM reasoning as latent-state trajectory formation rather than surface chain-of-thought, emphasizing the importance of disentangling different reasoning representations.

Contribution

It formalizes three hypotheses about LLM reasoning, argues for the primacy of latent states, and recommends new evaluation designs to better understand reasoning processes.

Findings

Current evidence most strongly supports reasoning as latent-state trajectories.

Empirical and mechanistic work should focus on latent-state dynamics.

Evaluation methods should disentangle surface traces, latent states, and compute.

Abstract

This position paper argues that large language model (LLM) reasoning should be studied as latent-state trajectory formation rather than as faithful surface chain-of-thought (CoT). This matters because claims about faithfulness, interpretability, reasoning benchmarks, and inference-time intervention all depend on what the field takes the primary object of reasoning to be. We ask what that object should be once three often-confounded factors are separated and formalize three competing hypotheses: H1, reasoning is primarily mediated by latent-state trajectories; H2, reasoning is primarily mediated by explicit surface CoT; and H0, most apparent reasoning gains are better explained by generic serial compute than by any privileged representational object. Reorganizing recent empirical, mechanistic, and survey work under this framework, and adding compute-audited worked exemplars that factorize surface traces, latent interventions, and matched budget expansions, we find that current evidence most strongly supports H1 as a default working hypothesis rather than as a task-independent verdict. We therefore make two recommendations: the field should treat latent-state dynamics as the default object of study for LLM reasoning, and it should evaluate reasoning with designs that explicitly disentangle surface traces, latent states, and serial compute.