Monte-Carlo Tree Search for Multi-Agent Pathfinding: Preliminary Results

TL;DR

This paper introduces a novel Monte-Carlo Tree Search variant tailored for multi-agent pathfinding on graphs, demonstrating improved performance over traditional heuristic methods through empirical evaluation.

Contribution

The paper presents an original MCTS variant specifically designed for multi-agent pathfinding, including a new reward computation method and a decomposition technique to enhance efficiency.

Findings

The proposed MCTS variant outperforms baseline heuristic search algorithms.

The reward mechanism effectively guides agents towards goal-reaching while avoiding collisions.

Decomposition reduces the search space, improving computational efficiency.

Abstract

In this work we study a well-known and challenging problem of Multi-agent Pathfinding, when a set of agents is confined to a graph, each agent is assigned a unique start and goal vertices and the task is to find a set of collision-free paths (one for each agent) such that each agent reaches its respective goal. We investigate how to utilize Monte-Carlo Tree Search (MCTS) to solve the problem. Although MCTS was shown to demonstrate superior performance in a wide range of problems like playing antagonistic games (e.g. Go, Chess etc.), discovering faster matrix multiplication algorithms etc., its application to the problem at hand was not well studied before. To this end we introduce an original variant of MCTS, tailored to multi-agent pathfinding. The crux of our approach is how the reward, that guides MCTS, is computed. Specifically, we use individual paths to assist the agents with the the goal-reaching behavior, while leaving them freedom to get off the track if it is needed to avoid collisions. We also use a dedicated decomposition technique to reduce the branching factor of the tree search procedure. Empirically we show that the suggested method outperforms the baseline planning algorithm that invokes heuristic search, e.g. A*, at each re-planning step.