Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion

TL;DR

This paper introduces a concurrent training framework for a control policy and state estimator in legged robots, enabling robust and dynamic locomotion across diverse terrains with high speeds.

Contribution

It presents a novel framework for simultaneous training of control and estimation networks, successfully transferring to real robots for versatile terrain traversal.

Findings

Able to traverse diverse terrains including hills, slippery, and bumpy surfaces.

Achieves speeds up to 3.75 m/s on flat ground and 3.54 m/s on slippery surfaces.

Demonstrates effective real-world transfer of trained networks.

Abstract

In this paper, we propose a locomotion training framework where a control policy and a state estimator are trained concurrently. The framework consists of a policy network which outputs the desired joint positions and a state estimation network which outputs estimates of the robot's states such as the base linear velocity, foot height, and contact probability. We exploit a fast simulation environment to train the networks and the trained networks are transferred to the real robot. The trained policy and state estimator are capable of traversing diverse terrains such as a hill, slippery plate, and bumpy road. We also demonstrate that the learned policy can run at up to 3.75 m/s on normal flat ground and 3.54 m/s on a slippery plate with the coefficient of friction of 0.22.