Investigating the Impact of Communication-Induced Action Space on Exploration of Unknown Environments with Decentralized Multi-Agent Reinforcement Learning

TL;DR

This paper proposes a communication-induced action space in decentralized multi-agent reinforcement learning to enhance exploration efficiency and mapping accuracy in unknown environments, validated through ROS2 simulations with TurtleBot3 robots.

Contribution

It introduces a novel communication-based action space and reward functions for improved exploration in D-MARL, with validation in simulated robotic environments.

Findings

Enhanced mapping efficiency and robustness

Reduced exploration overlap

Effective communication strategies in multi-agent exploration

Abstract

This paper introduces a novel enhancement to the Decentralized Multi-Agent Reinforcement Learning (D-MARL) exploration by proposing communication-induced action space to improve the mapping efficiency of unknown environments using homogeneous agents. Efficient exploration of large environments relies heavily on inter-agent communication as real-world scenarios are often constrained by data transmission limits, such as signal latency and bandwidth. Our proposed method optimizes each agent's policy using the heterogeneous-agent proximal policy optimization algorithm, allowing agents to autonomously decide whether to communicate or to explore, that is whether to share the locally collected maps or continue the exploration. We propose and compare multiple novel reward functions that integrate inter-agent communication and exploration, enhance mapping efficiency and robustness, and minimize exploration overlap. This article presents a framework developed in ROS2 to evaluate and validate the investigated architecture. Specifically, four TurtleBot3 Burgers have been deployed in a Gazebo-designed environment filled with obstacles to evaluate the efficacy of the trained policies in mapping the exploration arena.