Modeling Hierarchical Thinking in Large Reasoning Models

TL;DR

This paper introduces a finite state machine framework to interpret and analyze the hierarchical reasoning processes of large language models during chain-of-thought generation, revealing distinct reasoning patterns.

Contribution

It proposes a structured FSM approach to interpret LLM reasoning, enabling visualization and comparison of reasoning states and trajectories.

Findings

Identifies key reasoning states such as deduction and backtracking.

Provides a method to annotate and analyze reasoning trajectories.

Reveals differences in reasoning patterns across models.

Abstract

Large Language Models (LLMs) have demonstrated remarkable reasoning abilities when they generate step-by-step solutions, known as chain-of-thought (CoT) reasoning. When trained to using chain-of-thought reasoning examples, the resulting models (called Large Reasoning Models, or LRMs) appear to learn hierarchical thinking strategies similar to those used by humans. However, understanding LRMs emerging reasoning capabilities remains a difficult open problem, with many potential important applications including improving training and understanding robustness. In this paper, we adopt a memoryless Finite State Machine formulation to approximate LRM's emerging hierarchical reasoning dynamics as a structured, interpretable abstraction. We identify a small set of discrete reasoning states including - initialization, deduction, augmentation-strategy, uncertainty-estimation, backtracking, and final-conclusion that capture the high-level states present in the model's reasoning process. By annotating each step of a model's CoT with these states, we can represent the reasoning trajectory as a transition sequence through the state graph. This FSM formulation provides a systematic way to analyze, interpret and visualize how different models approach problems. We describe the FSM model, provide examples of CoT annotations under this scheme, and discuss how it can shed light on differences between available models in their approach to reasoning. Our results demonstrate that this FSM-based analysis reveals distinct reasoning patterns and potential shortcomings, offering a new lens to evaluate and improve LLM reasoning.