Data-Driven Latent Space Representation for Robust Bipedal Locomotion Learning

TL;DR

This paper introduces a data-driven approach that uses autoencoders to learn a low-dimensional, disentangled latent space for robust bipedal locomotion, improving over traditional methods in simulation.

Contribution

It combines autoencoder-based latent space learning with reinforcement learning to develop a robust, generalizable bipedal gait policy without heuristic or template models.

Findings

Latent variables correspond to different gaits and speeds.

The framework outperforms traditional template model-based approaches.

The learned policy generalizes well to unseen scenarios.

Abstract

This paper presents a novel framework for learning robust bipedal walking by combining a data-driven state representation with a Reinforcement Learning (RL) based locomotion policy. The framework utilizes an autoencoder to learn a low-dimensional latent space that captures the complex dynamics of bipedal locomotion from existing locomotion data. This reduced dimensional state representation is then used as states for training a robust RL-based gait policy, eliminating the need for heuristic state selections or the use of template models for gait planning. The results demonstrate that the learned latent variables are disentangled and directly correspond to different gaits or speeds, such as moving forward, backward, or walking in place. Compared to traditional template model-based approaches, our framework exhibits superior performance and robustness in simulation. The trained policy effectively tracks a wide range of walking speeds and demonstrates good generalization capabilities to unseen scenarios.