Learning a State Representation and Navigation in Cluttered and Dynamic Environments

TL;DR

This paper introduces a learning-based navigation system for quadrupedal robots that uses depth camera data to navigate cluttered environments without explicit mapping, combining state representation learning with reinforcement learning for efficient, real-world deployment.

Contribution

The work presents a novel pipeline that integrates unsupervised state representation learning with reinforcement learning for local navigation, enabling effective sim-to-real transfer in dynamic, cluttered environments.

Findings

Successful real-world navigation with a quadruped robot in cluttered environments.

Robust obstacle avoidance including dynamic obstacles unseen during training.

Efficient training in simulation taking only about a dozen minutes.

Abstract

In this work, we present a learning-based pipeline to realise local navigation with a quadrupedal robot in cluttered environments with static and dynamic obstacles. Given high-level navigation commands, the robot is able to safely locomote to a target location based on frames from a depth camera without any explicit mapping of the environment. First, the sequence of images and the current trajectory of the camera are fused to form a model of the world using state representation learning. The output of this lightweight module is then directly fed into a target-reaching and obstacle-avoiding policy trained with reinforcement learning. We show that decoupling the pipeline into these components results in a sample efficient policy learning stage that can be fully trained in simulation in just a dozen minutes. The key part is the state representation, which is trained to not only estimate the hidden state of the world in an unsupervised fashion, but also helps bridging the reality gap, enabling successful sim-to-real transfer. In our experiments with the quadrupedal robot ANYmal in simulation and in reality, we show that our system can handle noisy depth images, avoid dynamic obstacles unseen during training, and is endowed with local spatial awareness.