Learning Dynamic Bipedal Walking Across Stepping Stones

TL;DR

This paper presents a reinforcement learning-based approach enabling a bipedal robot to walk dynamically across stepping stones in real-world scenarios, bridging the gap between simulation and practical deployment.

Contribution

It introduces a novel learning framework combining RL, a predictive model, and real-time perception for robust, closed-loop bipedal walking over complex stepping-stone patterns.

Findings

Successful real-world demonstration on Cassie robot

Effective simulation-to-real transfer of walking controller

Benchmark set for evaluating stepping-stone walking performance

Abstract

In this work, we propose a learning approach for 3D dynamic bipedal walking when footsteps are constrained to stepping stones. While recent work has shown progress on this problem, real-world demonstrations have been limited to relatively simple open-loop, perception-free scenarios. Our main contribution is a more advanced learning approach that enables real-world demonstrations, using the Cassie robot, of closed-loop dynamic walking over moderately difficult stepping-stone patterns. Our approach first uses reinforcement learning (RL) in simulation to train a controller that maps footstep commands onto joint actions without any reference motion information. We then learn a model of that controller's capabilities, which enables prediction of feasible footsteps given the robot's current dynamic state. The resulting controller and model are then integrated with a real-time overhead camera system for detecting stepping stone locations. For evaluation, we develop a benchmark set of stepping stone patterns, which are used to test performance in both simulation and the real world. Overall, we demonstrate that sim-to-real learning is extremely promising for enabling dynamic locomotion over stepping stones. We also identify challenges remaining that motivate important future research directions.