Learning-Based Sparsification of Dynamic Graphs in Robotic Exploration Algorithms

TL;DR

This paper introduces a transformer-based reinforcement learning framework that effectively prunes dynamic graphs in robotic exploration, significantly reducing graph size and improving consistency across environments.

Contribution

It is the first to demonstrate the viability of RL for sparsifying dynamic graphs in robotic exploration algorithms, achieving substantial graph size reduction.

Findings

Graph size reduced by up to 96% using the learned policy.

The framework learns to associate pruning with exploration outcomes despite sparse rewards.

Pruning results in more consistent exploration across different environments.

Abstract

Many robotic exploration algorithms rely on graph structures for frontier-based exploration and dynamic path planning. However, these graphs grow rapidly, accumulating redundant information and impacting performance. We present a transformer-based framework trained with Proximal Policy Optimization (PPO) to prune these graphs during exploration, limiting their growth and reducing the accumulation of excess information. The framework was evaluated on simulations of a robotic agent using Rapidly Exploring Random Trees (RRT) to carry out frontier-based exploration, where the learned policy reduces graph size by up to 96%. We find preliminary evidence that our framework learns to associate pruning decisions with exploration outcomes despite sparse, delayed reward signals. We also observe that while intelligent pruning achieves a lower rate of exploration compared to baselines, it yields the lowest standard deviation, producing the most consistent exploration across varied environments. To the best of our knowledge, these results are the first suggesting the viability of RL in sparsification of dynamic graphs used in robotic exploration algorithms.