ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills

TL;DR

This paper introduces ALLSTEPS, a reinforcement learning approach with curriculum strategies to teach characters and robots to navigate complex stepping-stone terrains effectively, demonstrating robustness and plausibility across various simulations.

Contribution

It presents a curriculum-driven reinforcement learning method for stepping-stone locomotion, outperforming non-curriculum baselines in diverse simulated environments.

Findings

Curriculum strategies significantly improve learning efficiency.

The method produces robust and plausible stepping motions.

Effective across different character and robot models.

Abstract

Humans are highly adept at walking in environments with foot placement constraints, including stepping-stone scenarios where the footstep locations are fully constrained. Finding good solutions to stepping-stone locomotion is a longstanding and fundamental challenge for animation and robotics. We present fully learned solutions to this difficult problem using reinforcement learning. We demonstrate the importance of a curriculum for efficient learning and evaluate four possible curriculum choices compared to a non-curriculum baseline. Results are presented for a simulated human character, a realistic bipedal robot simulation and a monster character, in each case producing robust, plausible motions for challenging stepping stone sequences and terrains.