GraspDiffusion: Synthesizing Realistic Whole-body Hand-Object Interaction

TL;DR

GraspDiffusion is a new generative approach that synthesizes realistic full-body human-object interaction scenes by jointly modeling body and hand poses, improving over previous methods in realism and diversity.

Contribution

It introduces a novel generative method that constructs whole-body poses with controlled object placement, leveraging separate priors for body and hand poses to produce realistic interactions.

Findings

Successfully generates diverse, realistic human-object interaction scenes.

Outperforms previous methods in quality and diversity.

Effectively models intricate hand-object interactions.

Abstract

Recent generative models can synthesize high-quality images, but they often fail to generate humans interacting with objects using their hands. This arises mostly from the model's misunderstanding of such interactions and the hardships of synthesizing intricate regions of the body. In this paper, we propose \textbf{GraspDiffusion}, a novel generative method that creates realistic scenes of human-object interaction. Given a 3D object, GraspDiffusion constructs whole-body poses with control over the object's location relative to the human body, which is achieved by separately leveraging the generative priors for body and hand poses, optimizing them into a joint grasping pose. This pose guides the image synthesis to correctly reflect the intended interaction, creating realistic and diverse human-object interaction scenes. We demonstrate that GraspDiffusion can successfully tackle the relatively uninvestigated problem of generating full-bodied human-object interactions while outperforming previous methods. Our project page is available at https://yj7082126.github.io/graspdiffusion/