Motion-X++: A Large-Scale Multimodal 3D Whole-body Human Motion Dataset

TL;DR

Motion-X++ is a large-scale, multimodal 3D human motion dataset that enhances previous datasets with more detailed annotations, multiple modalities, and scalability, supporting various motion generation and analysis tasks.

Contribution

We developed an improved annotation pipeline and scaled up the dataset to include diverse modalities and extensive data, enabling advanced research in human motion understanding.

Findings

High annotation accuracy validated experimentally

Supports multiple downstream tasks including motion generation and pose estimation

Significantly larger and more comprehensive than previous datasets

Abstract

In this paper, we introduce Motion-X++, a large-scale multimodal 3D expressive whole-body human motion dataset. Existing motion datasets predominantly capture body-only poses, lacking facial expressions, hand gestures, and fine-grained pose descriptions, and are typically limited to lab settings with manually labeled text descriptions, thereby restricting their scalability. To address this issue, we develop a scalable annotation pipeline that can automatically capture 3D whole-body human motion and comprehensive textural labels from RGB videos and build the Motion-X dataset comprising 81.1K text-motion pairs. Furthermore, we extend Motion-X into Motion-X++ by improving the annotation pipeline, introducing more data modalities, and scaling up the data quantities. Motion-X++ provides 19.5M 3D whole-body pose annotations covering 120.5K motion sequences from massive scenes, 80.8K RGB videos, 45.3K audios, 19.5M frame-level whole-body pose descriptions, and 120.5K sequence-level semantic labels. Comprehensive experiments validate the accuracy of our annotation pipeline and highlight Motion-X++'s significant benefits for generating expressive, precise, and natural motion with paired multimodal labels supporting several downstream tasks, including text-driven whole-body motion generation,audio-driven motion generation, 3D whole-body human mesh recovery, and 2D whole-body keypoints estimation, etc.