No Global Plan in Chain-of-Thought: Uncover the Latent Planning Horizon of LLMs

TL;DR

This paper investigates the internal planning capabilities of Large Language Models during Chain-of-Thought reasoning, revealing a limited global planning horizon and proposing methods to improve uncertainty estimation and bypass recognition without performance loss.

Contribution

It introduces Tele-Lens, a probing method to analyze LLMs' internal states, and demonstrates that LLMs have a myopic planning horizon, leading to new approaches for uncertainty estimation and reasoning bypass detection.

Findings

LLMs exhibit a primarily incremental, myopic planning horizon.

A small subset of CoT positions can effectively estimate overall uncertainty.

Automatic CoT bypass recognition can be achieved without degrading performance.

Abstract

This work stems from prior complementary observations on the dynamics of Chain-of-Thought (CoT): Large Language Models (LLMs) is shown latent planning of subsequent reasoning prior to CoT emergence, thereby diminishing the significance of explicit CoT; whereas CoT remains critical for tasks requiring multi-step reasoning. To deepen the understanding between LLM's internal states and its verbalized reasoning trajectories, we investigate the latent planning strength of LLMs, through our probing method, Tele-Lens, applying to hidden states across diverse task domains. Our empirical results indicate that LLMs exhibit a myopic horizon, primarily conducting incremental transitions without precise global planning. Leveraging this characteristic, we propose a hypothesis on enhancing uncertainty estimation of CoT, which we validate that a small subset of CoT positions can effectively represent the uncertainty of the entire path. We further underscore the significance of exploiting CoT dynamics, and demonstrate that automatic recognition of CoT bypass can be achieved without performance degradation. Our code, data and models are released at https://github.com/lxucs/tele-lens.