Learning Robot Exploration Strategy with 4D Point-Clouds-like Information as Observations

TL;DR

This paper introduces a novel 4D point-clouds-like state representation and neural network approach for robot exploration, enabling better generalization to environments of varying sizes and reducing exploration distances.

Contribution

The paper proposes a new 4D point-clouds-like state representation and neural network model trained with reinforcement learning for scalable, environment-size-independent robot exploration strategies.

Findings

Outperforms five existing methods in exploration efficiency

Requires shorter average travel distances

Scales effectively to larger maps than training set sizes

Abstract

Being able to explore unknown environments is a requirement for fully autonomous robots. Many learning-based methods have been proposed to learn an exploration strategy. In the frontier-based exploration, learning algorithms tend to learn the optimal or near-optimal frontier to explore. Most of these methods represent the environments as fixed size images and take these as inputs to neural networks. However, the size of environments is usually unknown, which makes these methods fail to generalize to real world scenarios. To address this issue, we present a novel state representation method based on 4D point-clouds-like information, including the locations, frontier, and distance information. We also design a neural network that can process these 4D point-clouds-like information and generate the estimated value for each frontier. Then this neural network is trained using the typical reinforcement learning framework. We test the performance of our proposed method by comparing it with other five methods and test its scalability on a map that is much larger than maps in the training set. The experiment results demonstrate that our proposed method needs shorter average traveling distances to explore whole environments and can be adopted in maps with arbitrarily sizes.