MOGRAS: Human Motion with Grasping in 3D Scenes

TL;DR

This paper introduces MOGRAS, a large-scale dataset for realistic human full-body motion with grasping in 3D scenes, addressing the gap between scene-aware motion and precise grasping, and proposes a method to improve existing approaches.

Contribution

We present MOGRAS, a comprehensive dataset for human motion with grasping in 3D scenes, and propose a simple method to adapt existing models for scene-aware grasping tasks.

Findings

MOGRAS enables better benchmarking of scene-aware grasping methods.

Our method improves the realism and accuracy of human-scene interaction generation.

Extensive experiments validate the effectiveness of the dataset and proposed approach.

Abstract

Generating realistic full-body motion interacting with objects is critical for applications in robotics, virtual reality, and human-computer interaction. While existing methods can generate full-body motion within 3D scenes, they often lack the fidelity for fine-grained tasks like object grasping. Conversely, methods that generate precise grasping motions typically ignore the surrounding 3D scene. This gap, generating full-body grasping motions that are physically plausible within a 3D scene, remains a significant challenge. To address this, we introduce MOGRAS (Human MOtion with GRAsping in 3D Scenes), a large-scale dataset that bridges this gap. MOGRAS provides pre-grasping full-body walking motions and final grasping poses within richly annotated 3D indoor scenes. We leverage MOGRAS to benchmark existing full-body grasping methods and demonstrate their limitations in scene-aware generation. Furthermore, we propose a simple yet effective method to adapt existing approaches to work seamlessly within 3D scenes. Through extensive quantitative and qualitative experiments, we validate the effectiveness of our dataset and highlight the significant improvements our proposed method achieves, paving the way for more realistic human-scene interactions.