Deep Reinforcement Learning for Decentralized Multi-Robot Exploration With Macro Actions

TL;DR

This paper introduces MADE-Net, a multi-agent deep reinforcement learning approach enabling decentralized multi-robot exploration in complex environments despite communication dropouts, outperforming classical methods in efficiency and robustness.

Contribution

The paper presents the first macro action-based DRL framework for decentralized multi-robot exploration that effectively handles communication dropouts in unstructured environments.

Findings

MADE-Net outperforms classical and DRL benchmarks in exploration efficiency.

The method maintains robustness under varying communication dropout levels.

Scalability in 3D environments shows decreased exploration time with larger teams.

Abstract

Cooperative multi-robot teams need to be able to explore cluttered and unstructured environments while dealing with communication dropouts that prevent them from exchanging local information to maintain team coordination. Therefore, robots need to consider high-level teammate intentions during action selection. In this letter, we present the first Macro Action Decentralized Exploration Network (MADE-Net) using multi-agent deep reinforcement learning (DRL) to address the challenges of communication dropouts during multi-robot exploration in unseen, unstructured, and cluttered environments. Simulated robot team exploration experiments were conducted and compared against classical and DRL methods where MADE-Net outperformed all benchmark methods in terms of computation time, total travel distance, number of local interactions between robots, and exploration rate across various degrees of communication dropouts. A scalability study in 3D environments showed a decrease in exploration time with MADE-Net with increasing team and environment sizes. The experiments presented highlight the effectiveness and robustness of our method.