Multi-agent Path Finding for Timed Tasks using Evolutionary Games

TL;DR

This paper introduces a novel approach combining weighted automata and evolutionary game theory to improve multi-agent path finding for complex temporal tasks, outperforming traditional reinforcement learning in efficiency and scalability.

Contribution

The paper presents a new method using weighted automata for specifying trajectory-level objectives and applies evolutionary game theory to train multi-agent systems, surpassing RL in performance and speed.

Findings

Achieves nearly 30% shorter paths than RL methods.

Runs at least ten times faster than deep reinforcement learning.

Scales better with more agents compared to existing methods.

Abstract

Autonomous multi-agent systems such as hospital robots and package delivery drones often operate in highly uncertain environments and are expected to achieve complex temporal task objectives while ensuring safety. While learning-based methods such as reinforcement learning are popular methods to train single and multi-agent autonomous systems under user-specified and state-based reward functions, applying these methods to satisfy trajectory-level task objectives is a challenging problem. Our first contribution is the use of weighted automata to specify trajectory-level objectives, such that, maximal paths induced in the weighted automaton correspond to desired trajectory-level behaviors. We show how weighted automata-based specifications go beyond timeliness properties focused on deadlines to performance properties such as expeditiousness. Our second contribution is the use of evolutionary game theory (EGT) principles to train homogeneous multi-agent teams targeting homogeneous task objectives. We show how shared experiences of agents and EGT-based policy updates allow us to outperform state-of-the-art reinforcement learning (RL) methods in minimizing path length by nearly 30\% in large spaces. We also show that our algorithm is computationally faster than deep RL methods by at least an order of magnitude. Additionally our results indicate that it scales better with an increase in the number of agents as compared to other methods.