Make Planning Research Rigorous Again!
Michael Katz, Harsha Kokel, Christian Muise, Shirin Sohrabi, Sarath Sreedharan

TL;DR
This paper advocates for applying rigorous design and evaluation practices from traditional planning research to the development of large language model-based planners, emphasizing the importance of community insights and avoiding past pitfalls.
Contribution
It highlights the need to incorporate established planning community insights into LLM-based planner development to ensure rigor and avoid common pitfalls.
Findings
Traditional planning practices can improve LLM-based planner development.
Community insights are crucial for avoiding known pitfalls.
Rigorous evaluation methods are essential for progress.
Abstract
In over sixty years since its inception, the field of planning has made significant contributions to both the theory and practice of building planning software that can solve a never-before-seen planning problem. This was done through established practices of rigorous design and evaluation of planning systems. It is our position that this rigor should be applied to the current trend of work on planning with large language models. One way to do so is by correctly incorporating the insights, tools, and data from the automated planning community into the design and evaluation of LLM-based planners. The experience and expertise of the planning community are not just important from a historical perspective; the lessons learned could play a crucial role in accelerating the development of LLM-based planners. This position is particularly important in light of the abundance of recent works that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsUrban Planning and Governance · Geographic Information Systems Studies · 3D Modeling in Geospatial Applications
