BEDLAM2.0: Synthetic Humans and Cameras in Motion

TL;DR

BEDLAM2.0 introduces a comprehensive dataset with diverse, realistic human and camera motions, enhancing training and accuracy of 3D human motion estimation in world coordinates from video.

Contribution

The paper presents BEDLAM2.0, an improved dataset with greater diversity and realism, specifically designed to advance 3D human motion estimation in world coordinates.

Findings

BEDLAM2.0 outperforms BEDLAM in training 3D human pose models.

Models trained on BEDLAM2.0 show significantly improved accuracy.

The dataset includes diverse body shapes, clothing, hair, and environments.

Abstract

Inferring 3D human motion from video remains a challenging problem with many applications. While traditional methods estimate the human in image coordinates, many applications require human motion to be estimated in world coordinates. This is particularly challenging when there is both human and camera motion. Progress on this topic has been limited by the lack of rich video data with ground truth human and camera movement. We address this with BEDLAM2.0, a new dataset that goes beyond the popular BEDLAM dataset in important ways. In addition to introducing more diverse and realistic cameras and camera motions, BEDLAM2.0 increases diversity and realism of body shape, motions, clothing, hair, and 3D environments. Additionally, it adds shoes, which were missing in BEDLAM. BEDLAM has become a key resource for training 3D human pose and motion regressors today and we show that BEDLAM2.0 is significantly better, particularly for training methods that estimate humans in world coordinates. We compare state-of-the art methods trained on BEDLAM and BEDLAM2.0, and find that BEDLAM2.0 significantly improves accuracy over BEDLAM. For research purposes, we provide the rendered videos, ground truth body parameters, and camera motions. We also provide the 3D assets to which we have rights and links to those from third parties.