Why Solving Multi-agent Path Finding with Large Language Model has not Succeeded Yet

TL;DR

This paper investigates the challenges and limited success of using large language models for multi-agent path finding, highlighting the difficulties in multi-agent coordination and reasoning, and providing insights and hypotheses on why current approaches fall short.

Contribution

The paper provides a detailed analysis of the performance of LLMs on MAPF tasks, identifying key challenges and proposing hypotheses for their limited success in complex scenarios.

Findings

LLMs succeed on simple, obstacle-free MAPF scenarios

LLMs fail on complex maze and obstacle-laden MAPF scenarios

Experiments support the hypothesis that multi-agent coordination is a key challenge

Abstract

With the explosive influence caused by the success of large language models (LLM) like ChatGPT and GPT-4, there has been an extensive amount of recent work showing that foundation models can be used to solve a large variety of tasks. However, there is very limited work that shares insights on multi-agent planning. Multi-agent planning is different from other domains by combining the difficulty of multi-agent coordination and planning, and making it hard to leverage external tools to facilitate the reasoning needed. In this paper, we focus on the problem of multi-agent path finding (MAPF), which is also known as multi-robot route planning, and study the performance of solving MAPF with LLMs. We first show the motivating success on an empty room map without obstacles, then the failure to plan on the harder room map and maze map of the standard MAPF benchmark. We present our position on why directly solving MAPF with LLMs has not been successful yet, and we use various experiments to support our hypothesis. Based on our results, we discussed how researchers with different backgrounds could help with this problem from different perspectives.