Diffgrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model

TL;DR

Diffgrasp introduces a diffusion-based framework for generating realistic whole-body human-object interaction motions, effectively modeling hand-object relationships and grasping poses guided by object motion.

Contribution

It presents a novel joint modeling approach using diffusion models to synthesize coordinated whole-body motions with detailed grasping, surpassing prior static or incomplete methods.

Findings

Outperforms state-of-the-art in motion plausibility

Generates natural and coordinated grasping poses

Effectively models complex object and body interactions

Abstract

Generating high-quality whole-body human object interaction motion sequences is becoming increasingly important in various fields such as animation, VR/AR, and robotics. The main challenge of this task lies in determining the level of involvement of each hand given the complex shapes of objects in different sizes and their different motion trajectories, while ensuring strong grasping realism and guaranteeing the coordination of movement in all body parts. Contrasting with existing work, which either generates human interaction motion sequences without detailed hand grasping poses or only models a static grasping pose, we propose a simple yet effective framework that jointly models the relationship between the body, hands, and the given object motion sequences within a single diffusion model. To guide our network in perceiving the object's spatial position and learning more natural grasping poses, we introduce novel contact-aware losses and incorporate a data-driven, carefully designed guidance. Experimental results demonstrate that our approach outperforms the state-of-the-art method and generates plausible whole-body motion sequences.