LEAD: Breaking the No-Recovery Bottleneck in Long-Horizon Reasoning

TL;DR

LEAD introduces a method to improve long-horizon reasoning stability in LLMs by balancing decomposition with lookahead validation, enabling better problem-solving on complex algorithmic puzzles.

Contribution

The paper proposes LEAD, a novel approach that mitigates the no-recovery bottleneck in long-horizon reasoning by integrating short-horizon validation and overlapping rollouts.

Findings

LEAD enables solving Checkers Jumping up to complexity n=13.

Extreme decomposition fails beyond complexity n=11.

LEAD maintains stability and error correction in long-horizon tasks.

Abstract

Long-horizon execution in Large Language Models (LLMs) remains unstable even when high-level strategies are provided. Evaluating on controlled algorithmic puzzles, we demonstrate that while decomposition is essential for stability, extreme decomposition creates a "no-recovery bottleneck". We show that this bottleneck becomes critical due to highly non-uniform error distribution, where consistent errors on a few "hard" steps become irreversible. To address this, we propose Lookahead-Enhanced Atomic Decomposition (LEAD). By incorporating short-horizon future validation and aggregating overlapping rollouts, LEAD provides enough isolation to maintain stability while retaining enough local context to correct errors. This enables the o4-mini model to solve Checkers Jumping up to complexity $n=13$, whereas extreme decomposition fails beyond $n=11$.