Continuous Versatile Jumping Using Learned Action Residuals

TL;DR

This paper presents a hierarchical framework combining optimal control and reinforcement learning to enable quadrupedal robots to perform continuous, versatile jumps with improved stability and real-world deployment capabilities.

Contribution

It introduces a novel stance controller with learned residuals that enhances jumping stability and versatility, bridging simulation training and real-world application.

Findings

Achieved omni-directional jumps up to 50cm high

Performed 60cm forward jumps and 90-degree jump-turns

Framework successfully deployed on real robot

Abstract

Jumping is essential for legged robots to traverse through difficult terrains. In this work, we propose a hierarchical framework that combines optimal control and reinforcement learning to learn continuous jumping motions for quadrupedal robots. The core of our framework is a stance controller, which combines a manually designed acceleration controller with a learned residual policy. As the acceleration controller warm starts policy for efficient training, the trained policy overcomes the limitation of the acceleration controller and improves the jumping stability. In addition, a low-level whole-body controller converts the body pose command from the stance controller to motor commands. After training in simulation, our framework can be deployed directly to the real robot, and perform versatile, continuous jumping motions, including omni-directional jumps at up to 50cm high, 60cm forward, and jump-turning at up to 90 degrees. Please visit our website for more results: https://sites.google.com/view/learning-to-jump.