Learning to walk in confined spaces using 3D representation

TL;DR

This paper introduces a reinforcement learning-based control method for legged robots using 3D volumetric representations, enabling robust navigation in confined and unstructured environments through hierarchical policies.

Contribution

It presents a novel hierarchical policy framework combined with 3D representations and procedural terrain generation for improved legged robot locomotion in complex spaces.

Findings

Effective in simulation and real-world tests

Handles overhanging obstacles and rough terrain

Extends robot deployment to confined environments

Abstract

Legged robots have the potential to traverse complex terrain and access confined spaces beyond the reach of traditional platforms thanks to their ability to carefully select footholds and flexibly adapt their body posture while walking. However, robust deployment in real-world applications is still an open challenge. In this paper, we present a method for legged locomotion control using reinforcement learning and 3D volumetric representations to enable robust and versatile locomotion in confined and unstructured environments. By employing a two-layer hierarchical policy structure, we exploit the capabilities of a highly robust low-level policy to follow 6D commands and a high-level policy to enable three-dimensional spatial awareness for navigating under overhanging obstacles. Our study includes the development of a procedural terrain generator to create diverse training environments. We present a series of experimental evaluations in both simulation and real-world settings, demonstrating the effectiveness of our approach in controlling a quadruped robot in confined, rough terrain. By achieving this, our work extends the applicability of legged robots to a broader range of scenarios.