BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion

TL;DR

BeyondMimic introduces a versatile humanoid control framework using guided diffusion, enabling seamless composition of diverse agile motions and solving unseen tasks with high naturalness and zero-shot transfer to real hardware.

Contribution

It presents a unified diffusion-based model for scalable, natural, and versatile humanoid motion synthesis and task generalization, surpassing prior motion-specific approaches.

Findings

Achieves state-of-the-art human-like agility including flips and sprinting.

Enables zero-shot transfer of learned skills to real hardware.

Supports diverse downstream tasks like obstacle avoidance and teleoperation.

Abstract

The human-like form of humanoid robots positions them uniquely to achieve the agility and versatility in motor skills that humans possess. Learning from human demonstrations offers a scalable approach to acquiring these capabilities. However, prior works either produce unnatural motions or rely on motion-specific tuning to achieve satisfactory naturalness. Furthermore, these methods are often motion- or goal-specific, lacking the versatility to compose diverse skills, especially when solving unseen tasks. We present BeyondMimic, a framework that scales to diverse motions and carries the versatility to compose them seamlessly in tackling unseen downstream tasks. At heart, a compact motion-tracking formulation enables mastering a wide range of radically agile behaviors, including aerial cartwheels, spin-kicks, flip-kicks, and sprinting, with a single setup and shared hyperparameters, all while achieving state-of-the-art human-like performance. Moving beyond the mere imitation of existing motions, we propose a unified latent diffusion model that empowers versatile goal specification, seamless task switching, and dynamic composition of these agile behaviors. Leveraging classifier guidance, a diffusion-specific technique for test-time optimization toward novel objectives, our model extends its capability to solve downstream tasks never encountered during training, including motion inpainting, joystick teleoperation, and obstacle avoidance, and transfers these skills zero-shot to real hardware. This work opens new frontiers for humanoid robots by pushing the limits of scalable human-like motor skill acquisition from human motion and advancing seamless motion synthesis that achieves generalization and versatility beyond training setups.