Wide-Horizon Thinking and Simulation-Based Evaluation for Real-World LLM Planning with Multifaceted Constraints

TL;DR

This paper introduces MAoP, a wide-horizon planning approach for LLMs that considers multiple constraints simultaneously, and proposes Travel-Sim, a simulation-based benchmark for realistic evaluation of complex planning tasks.

Contribution

It presents MAoP for scalable, multi-aspect planning with LLMs and introduces Travel-Sim, a realistic simulation benchmark addressing causal dependencies in multi-constraint planning.

Findings

MAoP improves planning quality in complex, multi-constraint scenarios.

Travel-Sim provides a more realistic evaluation environment for planning tasks.

The approach enhances LLM capabilities for real-world, multifaceted planning.

Abstract

Unlike reasoning, which often entails a deep sequence of deductive steps, complex real-world planning is characterized by the need to synthesize a broad spectrum of parallel and potentially conflicting information and constraints. For example, in travel planning scenarios, it requires the integration of diverse real-world information and user preferences. While LLMs show promise, existing methods with long-horizon thinking struggle with handling multifaceted constraints, leading to suboptimal solutions. Motivated by the challenges of real-world travel planning, this paper introduces the Multiple Aspects of Planning (MAoP), empowering LLMs with "wide-horizon thinking" to solve planning problems with multifaceted constraints. Instead of direct planning, MAoP leverages the strategist to conduct pre-planning from various aspects and provide the planning blueprint for planners, enabling strong inference-time scalability by scaling aspects to consider various constraints. In addition, existing benchmarks for multi-constraint planning are flawed because they assess constraints in isolation, ignoring causal dependencies within the constraints, e.g, travel planning, where past activities dictate future itinerary. To address this, we propose Travel-Sim, an agent-based benchmark assessing plans via real-world simulation, thereby inherently resolving these causal dependencies. This paper advances LLM capabilities in complex planning and offers novel insights for evaluating sophisticated scenarios through simulation.