Passing Through Narrow Gaps with Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning approach for autonomous navigation of robots through narrow gaps, demonstrating high success rates in simulation and real-world tests, advancing robotic subterranean exploration.

Contribution

The paper presents a novel deep reinforcement learning method for gap navigation, including a gap behaviour policy and a goal-conditioned behaviour selection policy, trained in simulation and tested on real robots.

Findings

93% success rate in simulation with manual activation

63% success rate in simulation with autonomous activation

73% success rate on real robot with manual activation

Abstract

The U.S. Defense Advanced Research Projects Agency (DARPA) Subterranean Challenge requires teams of robots to traverse difficult and diverse underground environments. Traversing small gaps is one of the challenging scenarios that robots encounter. Imperfect sensor information makes it difficult for classical navigation methods, where behaviours require significant manual fine tuning. In this paper we present a deep reinforcement learning method for autonomously navigating through small gaps, where contact between the robot and the gap may be required. We first learn a gap behaviour policy to get through small gaps (only centimeters wider than the robot). We then learn a goal-conditioned behaviour selection policy that determines when to activate the gap behaviour policy. We train our policies in simulation and demonstrate their effectiveness with a large tracked robot in simulation and on the real platform. In simulation experiments, our approach achieves 93\% success rate when the gap behaviour is activated manually by an operator, and 63\% with autonomous activation using the behaviour selection policy. In real robot experiments, our approach achieves a success rate of 73\% with manual activation, and 40\% with autonomous behaviour selection. While we show the feasibility of our approach in simulation, the difference in performance between simulated and real world scenarios highlight the difficulty of direct sim-to-real transfer for deep reinforcement learning policies. In both the simulated and real world environments alternative methods were unable to traverse the gap.