Chasing Autonomy: Dynamic Retargeting and Control Guided RL for Performant and Controllable Humanoid Running

TL;DR

This paper introduces a reinforcement learning-based control pipeline for humanoid robots that enables dynamic, controllable, and long-duration running by retargeting human motions and optimizing reward structures, demonstrated on hardware.

Contribution

It presents a novel pipeline for retargeting human motions and optimizing rewards to improve humanoid running performance and controllability in real-world environments.

Findings

Achieved running speeds up to 3.3 m/s on hardware.

Demonstrated hundreds of meters of autonomous outdoor running.

Control-guided reward improves velocity tracking performance.

Abstract

Humanoid robots have the promise of locomoting like humans, including fast and dynamic running. Recently, reinforcement learning (RL) controllers that can mimic human motions have become popular as they can generate very dynamic behaviors, but they are often restricted to single motion play-back which hinders their deployment in long duration and autonomous locomotion. In this paper, we present a pipeline to dynamically retarget human motions through an optimization routine with hard constraints to generate improved periodic reference libraries from a single human demonstration. We then study the effect of both the reference motion and the reward structure on the reference and commanded velocity tracking, concluding that a goal-conditioned and control-guided reward which tracks dynamically optimized human data results in the best performance. We deploy the policy on hardware, demonstrating its speed and endurance by achieving running speeds of up to 3.3 m/s on a Unitree G1 robot and traversing hundreds of meters in real-world environments. Additionally, to demonstrate the controllability of the locomotion, we use the controller in a full perception and planning autonomy stack for obstacle avoidance while running outdoors.