Curriculum-Based Reinforcement Learning for Quadrupedal Jumping: A Reference-free Design

TL;DR

This paper demonstrates that quadrupedal robots can learn diverse jumping skills without reference trajectories by using curriculum-based reinforcement learning, achieving record distances and robustness on soft grounds.

Contribution

It introduces a reference-free curriculum reinforcement learning approach for quadrupedal jumping, enabling versatile and robust jumping behaviors without imitation.

Findings

Achieved a 90cm forward jump, surpassing previous records.

Successfully learned to jump across obstacles conditioned on environment parameters.

Robot reliably performs continuous jumps on soft grassy terrain.

Abstract

Deep reinforcement learning (DRL) has emerged as a promising solution to mastering explosive and versatile quadrupedal jumping skills. However, current DRL-based frameworks usually rely on pre-existing reference trajectories obtained by capturing animal motions or transferring experience from existing controllers. This work aims to prove that learning dynamic jumping is possible without relying on imitating a reference trajectory by leveraging a curriculum design. Starting from a vertical in-place jump, we generalize the learned policy to forward and diagonal jumps and, finally, we learn to jump across obstacles. Conditioned on the desired landing location, orientation, and obstacle dimensions, the proposed approach yields a wide range of omnidirectional jumping motions in real-world experiments. Particularly we achieve a 90cm forward jump, exceeding all previous records for similar robots reported in the existing literature. Additionally, the robot can reliably execute continuous jumping on soft grassy grounds, which is especially remarkable as such conditions were not included in the training stage. A supplementary video can be found on: https://www.youtube.com/watch?v=nRaMCrwU5X8. The code associated with this work can be found on: https://github.com/Vassil17/Curriculum-Quadruped-Jumping-DRL.