Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Example

TL;DR

This study evaluates the robustness and limitations of LLM-based agents in complex, long-horizon planning tasks using the TravelPlanner benchmark, highlighting challenges and proposing a feedback-aware fine-tuning method for improvement.

Contribution

The paper introduces a comprehensive analysis of LLM agents in real-world planning, identifying key failure modes and proposing Feedback-Aware Fine-Tuning (FAFT) to enhance performance.

Findings

LLMs struggle to focus on crucial parts of long contexts.

They have difficulty analyzing and providing feedback on long plans.

FAFT significantly improves LLM performance over standard fine-tuning.

Abstract

Large language models (LLMs) have brought autonomous agents closer to artificial general intelligence (AGI) due to their promising generalization and emergent capabilities. There is, however, a lack of studies on how LLM-based agents behave, why they could potentially fail, and how to improve them, particularly in demanding real-world planning tasks. In this paper, as an effort to fill the gap, we present our study using a realistic benchmark, TravelPlanner, where an agent must meet multiple constraints to generate accurate plans. We leverage this benchmark to address four key research questions: (1) are LLM agents robust enough to lengthy and noisy contexts when it comes to reasoning and planning? (2) can few-shot prompting adversely impact the performance of LLM agents in scenarios with long context? (3) can we rely on refinement to improve plans, and (4) can fine-tuning LLMs with both positive and negative feedback lead to further improvement? Our comprehensive experiments indicate that, firstly, LLMs often fail to attend to crucial parts of a long context, despite their ability to handle extensive reference information and few-shot examples; secondly, they still struggle with analyzing the long plans and cannot provide accurate feedback for refinement; thirdly, we propose Feedback-Aware Fine-Tuning (FAFT), which leverages both positive and negative feedback, resulting in substantial gains over Supervised Fine-Tuning (SFT). Our findings offer in-depth insights to the community on various aspects related to real-world planning applications.