Human-Centered Planning

TL;DR

This paper introduces LLMPlan, an LLM-based planner for daily scheduling that incorporates self-reflection and natural language constraints, outperforming symbolic planners in user satisfaction despite similar constraint satisfaction.

Contribution

The paper presents LLMPlan, a novel LLM-based planning approach with self-reflection and natural language constraint handling, bridging the gap between symbolic and neural planning methods.

Findings

LLMPlan achieves 2% performance difference in constraint satisfaction compared to symbolic planners.

LLMPlan significantly outperforms symbolic planners in user satisfaction (70.5% vs. 40.4%).

LLMPlan effectively incorporates vague natural language constraints into planning.

Abstract

LLMs have recently made impressive inroads on tasks whose output is structured, such as coding, robotic planning and querying databases. The vision of creating AI-powered personal assistants also involves creating structured outputs, such as a plan for one's day, or for an overseas trip. Here, since the plan is executed by a human, the output doesn't have to satisfy strict syntactic constraints. A useful assistant should also be able to incorporate vague constraints specified by the user in natural language. This makes LLMs an attractive option for planning. We consider the problem of planning one's day. We develop an LLM-based planner (LLMPlan) extended with the ability to self-reflect on its output and a symbolic planner (SymPlan) with the ability to translate text constraints into a symbolic representation. Despite no formal specification of constraints, we find that LLMPlan performs explicit constraint satisfaction akin to the traditional symbolic planners on average (2% performance difference), while retaining the reasoning of implicit requirements. Consequently, LLM-based planners outperform their symbolic counterparts in user satisfaction (70.5% vs. 40.4%) during interactive evaluation with 40 users.