CQLite: Communication-Efficient Multi-Robot Exploration Using Coverage-biased Distributed Q-Learning

TL;DR

CQLite introduces a communication-efficient distributed Q-learning approach for multi-robot exploration, significantly reducing data sharing costs while maintaining rapid convergence and thorough coverage, outperforming existing methods.

Contribution

It presents a novel distributed Q-learning method that minimizes communication overhead in multi-robot exploration through selective sharing and map merging.

Findings

Over 2x reduction in communication and computation costs.

Achieved faster convergence and better coverage than existing methods.

Demonstrated effectiveness on simulated indoor maps with multiple robots.

Abstract

Frontier exploration and reinforcement learning have historically been used to solve the problem of enabling many mobile robots to autonomously and cooperatively explore complex surroundings. These methods need to keep an internal global map for navigation, but they do not take into consideration the high costs of communication and information sharing between robots. This study offers CQLite, a novel distributed Q-learning technique designed to minimize data communication overhead between robots while achieving rapid convergence and thorough coverage in multi-robot exploration. The proposed CQLite method uses ad hoc map merging, and selectively shares updated Q-values at recently identified frontiers to significantly reduce communication costs. The theoretical analysis of CQLite's convergence and efficiency, together with extensive numerical verification on simulated indoor maps utilizing several robots, demonstrates the method's novelty. With over 2x reductions in computation and communication alongside improved mapping performance, CQLite outperformed cutting-edge multi-robot exploration techniques like Rapidly Exploring Random Trees and Deep Reinforcement Learning. Related codes are open-sourced at \url{https://github.com/herolab-uga/cqlite}.