Hilbert-Augmented Reinforcement Learning for Scalable Multi-Robot Coverage and Exploration

TL;DR

This paper introduces a novel multi-robot coverage framework that incorporates Hilbert space-filling curves into reinforcement learning, enhancing exploration efficiency, scalability, and trajectory feasibility for resource-limited robots in complex environments.

Contribution

The work integrates Hilbert space-filling priors into decentralized RL algorithms, enabling scalable, efficient coverage and exploration with feasible trajectories for multi-robot systems.

Findings

Improved coverage efficiency and reduced redundancy.

Faster convergence in sparse-reward environments.

Successful deployment on real legged robots.

Abstract

We present a coverage framework that integrates Hilbert space-filling priors into decentralized multi-robot learning and execution. We augment DQN and PPO with Hilbert-based spatial indices to structure exploration and reduce redundancy in sparse-reward environments, and we evaluate scalability in multi-robot grid coverage. We further describe a waypoint interface that converts Hilbert orderings into curvature-bounded, time-parameterized SE(2) trajectories (planar (x, y, {\theta})), enabling onboard feasibility on resource-constrained robots. Experiments show improvements in coverage efficiency, redundancy, and convergence speed over DQN/PPO baselines. In addition, we validate the approach on a Boston Dynamics Spot legged robot, executing the generated trajectories in indoor environments and observing reliable coverage with low redundancy. These results indicate that geometric priors improve autonomy and scalability for swarm and legged robotics.