LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environment

TL;DR

LaserHuman is a new dataset with diverse, real-world human motions and natural language descriptions in 3D environments, enabling advanced scene-aware motion generation research.

Contribution

The paper introduces LaserHuman, a comprehensive dataset for scene-aware human motion generation, and proposes a multi-conditional diffusion model achieving state-of-the-art results.

Findings

LaserHuman enables improved scene-aware motion generation.

The diffusion model achieves state-of-the-art performance.

Rich annotations facilitate diverse research applications.

Abstract

Language-guided scene-aware human motion generation has great significance for entertainment and robotics. In response to the limitations of existing datasets, we introduce LaserHuman, a pioneering dataset engineered to revolutionize Scene-Text-to-Motion research. LaserHuman stands out with its inclusion of genuine human motions within 3D environments, unbounded free-form natural language descriptions, a blend of indoor and outdoor scenarios, and dynamic, ever-changing scenes. Diverse modalities of capture data and rich annotations present great opportunities for the research of conditional motion generation, and can also facilitate the development of real-life applications. Moreover, to generate semantically consistent and physically plausible human motions, we propose a multi-conditional diffusion model, which is simple but effective, achieving state-of-the-art performance on existing datasets.