PRISM: A Dual View of LLM Reasoning through Semantic Flow and Latent Computation

TL;DR

PRISM is a diagnostic framework that jointly analyzes the semantic flow and internal computations of LLM reasoning, revealing patterns in how models process complex tasks and how prompting influences reasoning behavior.

Contribution

It introduces a unified method for analyzing LLM reasoning across steps and layers, uncovering systematic patterns and effects of prompting on internal reasoning processes.

Findings

Failed reasoning trajectories often get stuck in verification loops.

Prompting changes internal reasoning behaviors beyond accuracy.

PRISM reveals distinct modes like overthinking and premature commitment.

Abstract

Large language models (LLMs) solve complex problems by generating multi-step reasoning traces. Yet these traces are typically analyzed from only one of two perspectives: the sequence of tokens across different reasoning steps in the generated text, or the hidden-state vectors across model layers within one step. We introduce PRISM (Probabilistic Reasoning Inspection through Semantic and Implicit Modeling), a framework and diagnostic tool for jointly analyzing both levels, providing a unified view of how reasoning evolves across steps and layers. Across multiple reasoning models and benchmarks, PRISM uncovers systematic patterns in the reasoning process, showing that failed trajectories are more likely to become trapped in unproductive verification loops and further diverge into distinct modes such as overthinking and premature commitment, which behave differently once a candidate answer is reached. It further reveals how prompting reshapes reasoning behavior beyond aggregate accuracy by altering both semantic transitions and internal computational patterns. By modeling reasoning trajectories as structured processes, PRISM makes these behaviors observable and analyzable rather than relying solely on final-task accuracy. Taken together, these insights position PRISM as a practical tool for analyzing and diagnosing reasoning processes in LLMs.