UniTracker: Learning Universal Whole-Body Motion Tracker for Humanoid Robots

TL;DR

UniTracker is a three-stage training framework that enables humanoid robots to learn and adapt to a wide range of human motions with high accuracy and robustness, combining teacher policies, CVAE models, and fast adaptation modules.

Contribution

The paper introduces a novel three-stage training method using CVAE to create a universal, scalable motion tracker for humanoid robots that generalizes to unseen behaviors.

Findings

Effective in both simulation and real-world tests.

Achieves high motion diversity and tracking accuracy.

Demonstrates robustness and scalability in deployment.

Abstract

Achieving expressive and generalizable whole-body motion control is essential for deploying humanoid robots in real-world environments. In this work, we propose UniTracker, a three-stage training framework that enables robust and scalable motion tracking across a wide range of human behaviors. In the first stage, we train a teacher policy with privileged observations to generate high-quality actions. In the second stage, we introduce a Conditional Variational Autoencoder (CVAE) to model a universal student policy that can be deployed directly on real hardware. The CVAE structure allows the policy to learn a global latent representation of motion, enhancing generalization to unseen behaviors and addressing the limitations of standard MLP-based policies under partial observations. Unlike pure MLPs that suffer from drift in global attributes like orientation, our CVAE-student policy incorporates global intent during training by aligning a partial-observation prior to the full-observation encoder. In the third stage, we introduce a fast adaptation module that fine-tunes the universal policy on harder motion sequences that are difficult to track directly. This adaptation can be performed both for single sequences and in batch mode, further showcasing the flexibility and scalability of our approach. We evaluate UniTracker in both simulation and real-world settings using a Unitree G1 humanoid, demonstrating strong performance in motion diversity, tracking accuracy, and deployment robustness.