Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs

TL;DR

This paper introduces a scalable, real-time deep reinforcement learning approach using graph neural networks for autonomous exploration, enabling efficient mapping and information gathering in unknown environments.

Contribution

It presents a novel GNN-DRL framework for belief space planning that outperforms traditional methods in real-time autonomous exploration tasks.

Findings

Achieves accurate mapping with high information gain

Operates efficiently in real-time in large environments

Outperforms existing belief space planning methods

Abstract

We consider an autonomous exploration problem in which a range-sensing mobile robot is tasked with accurately mapping the landmarks in an a priori unknown environment efficiently in real-time; it must choose sensing actions that both curb localization uncertainty and achieve information gain. For this problem, belief space planning methods that forward-simulate robot sensing and estimation may often fail in real-time implementation, scaling poorly with increasing size of the state, belief and action spaces. We propose a novel approach that uses graph neural networks (GNNs) in conjunction with deep reinforcement learning (DRL), enabling decision-making over graphs containing exploration information to predict a robot's optimal sensing action in belief space. The policy, which is trained in different random environments without human intervention, offers a real-time, scalable decision-making process whose high-performance exploratory sensing actions yield accurate maps and high rates of information gain.