RTAW: An Attention Inspired Reinforcement Learning Method for Multi-Robot Task Allocation in Warehouse Environments

TL;DR

This paper introduces RTAW, a reinforcement learning algorithm with attention mechanisms for multi-robot task allocation in warehouses, achieving up to 14% improvement in total travel delay over existing methods.

Contribution

It proposes a novel deep multi-agent reinforcement learning method with attention-inspired policy architecture that scales to large numbers of robots and tasks.

Findings

Up to 14% reduction in total travel delay compared to state-of-the-art methods.

Effective scalability demonstrated with up to 1000 robots in simulations.

Improved makespan for large task lists in complex warehouse layouts.

Abstract

We present a novel reinforcement learning based algorithm for multi-robot task allocation problem in warehouse environments. We formulate it as a Markov Decision Process and solve via a novel deep multi-agent reinforcement learning method (called RTAW) with attention inspired policy architecture. Hence, our proposed policy network uses global embeddings that are independent of the number of robots/tasks. We utilize proximal policy optimization algorithm for training and use a carefully designed reward to obtain a converged policy. The converged policy ensures cooperation among different robots to minimize total travel delay (TTD) which ultimately improves the makespan for a sufficiently large task-list. In our extensive experiments, we compare the performance of our RTAW algorithm to state of the art methods such as myopic pickup distance minimization (greedy) and regret based baselines on different navigation schemes. We show an improvement of upto 14% (25-1000 seconds) in TTD on scenarios with hundreds or thousands of tasks for different challenging warehouse layouts and task generation schemes. We also demonstrate the scalability of our approach by showing performance with up to $1000$ robots in simulations.