Custom Sine Waves Are Enough for Imitation Learning of Bipedal Gaits with Different Styles

TL;DR

This paper demonstrates that simple sine wave references can effectively enable reinforcement learning of diverse bipedal gaits on the Cassie robot without expert knowledge, simplifying the design process.

Contribution

It introduces a novel approach using sine wave foot height references for learning bipedal locomotion, allowing style customization and reducing reliance on expert insights.

Findings

Successful learning of walking gait with simple sine wave references

Ability to customize gait styles through sine wave parameters

Efficient end-to-end learning without explicit model knowledge

Abstract

Not until recently, robust bipedal locomotion has been achieved through reinforcement learning. However, existing implementations rely heavily on insights and efforts from human experts, which is costly for the iterative design of robot systems. Also, styles of the learned motion are strictly limited to that of the reference. In this paper, we propose a new way to learn bipedal locomotion from a simple sine wave as the reference for foot heights. With the naive human insight that the two feet should be lifted up alternatively and periodically, we experimentally demonstrate on the Cassie robot that, a simple reward function is able to make the robot learn to walk end-to-end and efficiently without any explicit knowledge of the model. With custom sine waves, the learned gait pattern can also have customized styles. Codes are released at github.com/WooQi57/sin-cassie-rl.