SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search

TL;DR

SPIRAL introduces a novel framework embedding three specialized LLM agents into an MCTS loop, enabling guided, reflective, and grounded planning that significantly improves complex task performance and efficiency.

Contribution

The paper presents SPIRAL, a new LLM planning framework that integrates a Planner, Simulator, and Critic into MCTS, enhancing reasoning robustness and accuracy over existing methods.

Findings

Achieves 83.6% accuracy on DailyLifeAPIs, surpassing previous methods by over 16 percentage points.

Outperforms Chain-of-Thought and other state-of-the-art agents in complex planning tasks.

Demonstrates improved token efficiency and self-correcting reasoning capabilities.

Abstract

Large Language Models (LLMs) often falter at complex planning tasks that require exploration and self-correction, as their linear reasoning process struggles to recover from early mistakes. While search algorithms like Monte Carlo Tree Search (MCTS) can explore alternatives, they are often ineffective when guided by sparse rewards and fail to leverage the rich semantic capabilities of LLMs. We introduce SPIRAL (Symbolic LLM Planning via Grounded and Reflective Search), a novel framework that embeds a cognitive architecture of three specialized LLM agents into an MCTS loop. SPIRAL's key contribution is its integrated planning pipeline where a Planner proposes creative next steps, a Simulator grounds the search by predicting realistic outcomes, and a Critic provides dense reward signals through reflection. This synergy transforms MCTS from a brute-force search into a guided, self-correcting reasoning process. On the DailyLifeAPIs and HuggingFace datasets, SPIRAL consistently outperforms the default Chain-of-Thought planning method and other state-of-the-art agents. More importantly, it substantially surpasses other state-of-the-art agents; for example, SPIRAL achieves 83.6% overall accuracy on DailyLifeAPIs, an improvement of over 16 percentage points against the next-best search framework, while also demonstrating superior token efficiency. Our work demonstrates that structuring LLM reasoning as a guided, reflective, and grounded search process yields more robust and efficient autonomous planners. The source code, full appendices, and all experimental data are available for reproducibility at the official project repository.