Latent Space Reinforcement Learning for Multi-Robot Exploration

TL;DR

This paper presents a scalable multi-robot exploration method using latent space reinforcement learning, autoencoders for environment representation, and a novel procedural environment generator, demonstrating robustness and generalization in complex, communication-limited scenarios.

Contribution

It introduces a latent space reinforcement learning framework with autoencoders and a new procedural environment generator for multi-robot exploration.

Findings

Scales effectively with the number of agents.

Generalizes well to unseen environments.

Remains robust under communication constraints.

Abstract

Autonomous mapping of unknown environments is a critical challenge, particularly in scenarios where time is limited. Multi-agent systems can enhance efficiency through collaboration, but the scalability of motion-planning algorithms remains a key limitation. Reinforcement learning has been explored as a solution, but existing approaches are constrained by the limited input size required for effective learning, restricting their applicability to discrete environments. This work addresses that limitation by leveraging autoencoders to perform dimensionality reduction, compressing high-fidelity occupancy maps into latent state vectors while preserving essential spatial information. Additionally, we introduce a novel procedural generation algorithm based on Perlin noise, designed to generate topologically complex training environments that simulate asteroid fields, caves and forests. These environments are used for training the autoencoder and the navigation algorithm using a hierarchical deep reinforcement learning framework for decentralized coordination. We introduce a weighted consensus mechanism that modulates reliance on shared data via a tuneable trust parameter, ensuring robustness to accumulation of errors. Experimental results demonstrate that the proposed system scales effectively with number of agents and generalizes well to unfamiliar, structurally distinct environments and is resilient in communication-constrained settings.