Joint Diffusion for Universal Hand-Object Grasp Generation

TL;DR

This paper introduces a unified diffusion model that generates plausible and diverse hand-object grasps, leveraging large-scale object datasets to improve generalization to unseen shapes in animation and robotics.

Contribution

It proposes the Joint Hand-Object Diffusion (JHOD) model that unifies hand and object representation for grasp generation, enhancing diversity and generalization.

Findings

Generates plausible and diverse grasps both conditionally and unconditionally.

Leverages large-scale object datasets for better generalization to unseen shapes.

Achieves good visual plausibility and diversity in experiments.

Abstract

Predicting and generating human hand grasp over objects is critical for animation and robotic tasks. In this work, we focus on generating both the hand and objects in a grasp by a single diffusion model. Our proposed Joint Hand-Object Diffusion (JHOD) models the hand and object in a unified latent representation. It uses the hand-object grasping data to learn to accommodate hand and object to form plausible grasps. Also, to enforce the generalizability over diverse object shapes, it leverages large-scale object datasets to learn an inclusive object latent embedding. With or without a given object as an optional condition, the diffusion model can generate grasps unconditionally or conditional to the object. Compared to the usual practice of learning object-conditioned grasp generation from only hand-object grasp data, our method benefits from more diverse object data used for training to handle grasp generation more universally. According to both qualitative and quantitative experiments, both conditional and unconditional generation of hand grasp achieves good visual plausibility and diversity. With the extra inclusiveness of object representation learned from large-scale object datasets, the proposed method generalizes well to unseen object shapes.