Platform-Agnostic Reinforcement Learning Framework for Safe Exploration of Cluttered Environments with Graph Attention

TL;DR

This paper presents a platform-agnostic reinforcement learning framework that combines graph neural networks and safety filters to enable efficient and safe autonomous exploration in obstacle-rich environments, suitable for real-world robotic deployment.

Contribution

It introduces a novel reinforcement learning framework integrating a graph neural network policy with a safety filter for safe exploration in cluttered environments, trained with a new reward function.

Findings

Achieves efficient exploration with minimal safety interventions.

Demonstrates safety and efficiency in both simulations and real-world experiments.

Outperforms baseline methods in cluttered space exploration.

Abstract

Autonomous exploration of obstacle-rich spaces requires strategies that ensure efficiency while guaranteeing safety against collisions with obstacles. This paper investigates a novel platform-agnostic reinforcement learning framework that integrates a graph neural network-based policy for next-waypoint selection, with a safety filter ensuring safe mobility. Specifically, the neural network is trained using reinforcement learning through the Proximal Policy Optimization (PPO) algorithm to maximize exploration efficiency while minimizing safety filter interventions. Henceforth, when the policy proposes an infeasible action, the safety filter overrides it with the closest feasible alternative, ensuring consistent system behavior. In addition, this paper introduces a reward function shaped by a potential field that accounts for both the agent's proximity to unexplored regions and the expected information gain from reaching them. The proposed framework combines the adaptability of reinforcement learning-based exploration policies with the reliability provided by explicit safety mechanisms. This feature plays a key role in enabling the deployment of learning-based policies on robotic platforms operating in real-world environments. Extensive evaluations in both simulations and experiments performed in a lab environment demonstrate that the approach achieves efficient and safe exploration in cluttered spaces.