GMT: General Motion Tracking for Humanoid Whole-Body Control

TL;DR

This paper introduces GMT, a scalable framework with adaptive sampling and a mixture-of-experts architecture, enabling humanoid robots to accurately track diverse motions in real-world scenarios.

Contribution

GMT is a novel unified policy framework that improves motion tracking for humanoid robots through adaptive sampling and a mixture-of-experts design.

Findings

Achieves state-of-the-art performance in diverse motion tracking tasks

Effective in both simulation and real-world environments

Outperforms existing methods in general motion tracking

Abstract

The ability to track general whole-body motions in the real world is a useful way to build general-purpose humanoid robots. However, achieving this can be challenging due to the temporal and kinematic diversity of the motions, the policy's capability, and the difficulty of coordination of the upper and lower bodies. To address these issues, we propose GMT, a general and scalable motion-tracking framework that trains a single unified policy to enable humanoid robots to track diverse motions in the real world. GMT is built upon two core components: an Adaptive Sampling strategy and a Motion Mixture-of-Experts (MoE) architecture. The Adaptive Sampling automatically balances easy and difficult motions during training. The MoE ensures better specialization of different regions of the motion manifold. We show through extensive experiments in both simulation and the real world the effectiveness of GMT, achieving state-of-the-art performance across a broad spectrum of motions using a unified general policy. Videos and additional information can be found at https://gmt-humanoid.github.io.