Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning

TL;DR

This paper introduces a method to analyze LLM reasoning traces by extracting search trees, revealing that LLMs rely on shallow search and myopic planning, contrasting with human deep planning.

Contribution

The work presents a novel framework for characterizing LLM planning through search tree extraction and demonstrates key differences from human planning behaviors.

Findings

LLMs' search is shallower than humans'

Performance correlates with search breadth, not depth

Move choices are driven by shallow nodes, not deep lookahead

Abstract

Large language models (LLMs), especially reasoning models, generate extended chain-of-thought (CoT) reasoning that often contains explicit deliberation over future outcomes. Yet whether this deliberation constitutes genuine planning, how it is structured, and what aspects of it drive performance remain poorly understood. In this work, we introduce a new method to characterize LLM planning by extracting and quantifying search trees from reasoning traces in the four-in-a-row board game. By fitting computational models on the extracted search trees, we characterize how plans are structured and how they influence move decisions. We find that LLMs' search is shallower than humans', and that performance is predicted by search breadth rather than depth. Most strikingly, although LLMs expand deep nodes in their traces, their move choices are best explained by a myopic model that ignores those nodes entirely. A causal intervention study where we selectively prune CoT paragraphs further suggests that move selection is driven predominantly by shallow rather than deep nodes. These patterns contrast with human planning, where performance is driven primarily by deep search. Together, our findings reveal a key difference between LLM and human planning: while human expertise is driven by deeper search, LLMs do not act on deep lookahead. This dissociation offers targeted guidance for aligning LLM and human planning. More broadly, our framework provides a generalizable approach for interpreting the structure of LLM planning across strategic domains.