Tracking Progress in Multi-Agent Path Finding

TL;DR

This paper introduces new methodological and visualization tools to measure and compare progress in Multi-Agent Path Finding (MAPF), aiming to standardize evaluation and foster further research in the field.

Contribution

It presents tools that enable large-scale comparison of MAPF algorithms, helping to establish clear performance indicators and lower entry barriers for new researchers.

Findings

Provides a framework for standardized MAPF evaluation

Facilitates large-scale comparison of MAPF solvers

Enhances understanding of MAPF progress and challenges

Abstract

Multi-Agent Path Finding (MAPF) is an important core problem for many new and emerging industrial applications. Many works appear on this topic each year, and a large number of substantial advancements and performance improvements have been reported. Yet measuring overall progress in MAPF is difficult: there are many potential competitors, and the computational burden for comprehensive experimentation is prohibitively large. Moreover, detailed data from past experimentation is usually unavailable. In this work, we introduce a set of methodological and visualisation tools which can help the community establish clear indicators for state-of-the-art MAPF performance and which can facilitate large-scale comparisons between MAPF solvers. Our objectives are to lower the barrier of entry for new researchers and to further promote the study of MAPF, since progress in the area and the main challenges are made much clearer.