Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agents

TL;DR

This paper analyzes why LLM-based agents struggle with long-horizon planning, identifying a fundamental mismatch in reasoning strategies, and introduces FLARE, a planning method that improves long-term decision making and outperforms standard reasoning approaches.

Contribution

The paper introduces FLARE, a planning-centric approach that incorporates explicit lookahead and value propagation, addressing reasoning failures in long-horizon decision making for LLM agents.

Findings

FLARE improves task performance across multiple benchmarks.

FLARE enables LLM agents to outperform GPT-4 with standard reasoning.

Reasoning-based policies tend to cause myopic commitments over long horizons.

Abstract

Large language model (LLM)-based agents exhibit strong step-by-step reasoning capabilities over short horizons, yet often fail to sustain coherent behavior over long planning horizons. We argue that this failure reflects a fundamental mismatch: step-wise reasoning induces a form of step-wise greedy policy that is adequate for short horizons but fails in long-horizon planning, where early actions must account for delayed consequences. From this planning-centric perspective, we study LLM-based agents in deterministic, fully structured environments with explicit state transitions and evaluation signals. Our analysis reveals a core failure mode of reasoning-based policies: locally optimal choices induced by step-wise scoring lead to early myopic commitments that are systematically amplified over time and difficult to recover from. We introduce FLARE (Future-aware Lookahead with Reward Estimation) as a minimal instantiation of future-aware planning to enforce explicit lookahead, value propagation, and limited commitment in a single model, allowing downstream outcomes to influence early decisions. Across multiple benchmarks, agent frameworks, and LLM backbones, FLARE consistently improves task performance and planning-level behavior, frequently allowing LLaMA-8B with FLARE to outperform GPT-4o with standard step-by-step reasoning. These results establish a clear distinction between reasoning and planning.