Jump-Start Reinforcement Learning with Self-Evolving Priors for Extreme Monopedal Locomotion

TL;DR

This paper introduces JumpER, a reinforcement learning framework that uses self-evolving priors and staged curriculum learning to enable quadruped robots to perform robust monopedal hopping across challenging terrains.

Contribution

The paper presents a novel RL training method that progressively refines policies through self-evolving priors, achieving stable learning without external experts or handcrafted rewards.

Findings

Enables quadruped robots to perform monopedal hopping on complex terrains.

Successfully handles wide gaps, irregular stairs, and variable stepping stones.

Achieves robust locomotion under extreme underactuation and terrain challenges.

Abstract

Reinforcement learning (RL) has shown great potential in enabling quadruped robots to perform agile locomotion. However, directly training policies to simultaneously handle dual extreme challenges, i.e., extreme underactuation and extreme terrains, as in monopedal hopping tasks, remains highly challenging due to unstable early-stage interactions and unreliable reward feedback. To address this, we propose JumpER (jump-start reinforcement learning via self-evolving priors), an RL training framework that structures policy learning into multiple stages of increasing complexity. By dynamically generating self-evolving priors through iterative bootstrapping of previously learned policies, JumpER progressively refines and enhances guidance, thereby stabilizing exploration and policy optimization without relying on external expert priors or handcrafted reward shaping. Specifically, when integrated with a structured three-stage curriculum that incrementally evolves action modality, observation space, and task objective, JumpER enables quadruped robots to achieve robust monopedal hopping on unpredictable terrains for the first time. Remarkably, the resulting policy effectively handles challenging scenarios that traditional methods struggle to conquer, including wide gaps up to 60 cm, irregularly spaced stairs, and stepping stones with distances varying from 15 cm to 35 cm. JumpER thus provides a principled and scalable approach for addressing locomotion tasks under the dual challenges of extreme underactuation and extreme terrains.