RegionGrasp: A Novel Task for Contact Region Controllable Hand Grasp Generation

TL;DR

This paper introduces RegionGrasp, a new task for generating diverse, contact-region-specific hand grasps on 3D objects, and proposes a novel model combining surface-aware encoding and interaction modeling.

Contribution

The paper presents RegionGrasp-CVAE, a novel framework that incorporates contact region awareness and interaction features for controllable hand grasp generation.

Findings

Outperforms existing methods in qualitative and quantitative evaluations.

Effectively captures surface geometry and interaction details.

Enables diverse and natural hand grasp generation based on contact regions.

Abstract

Can machine automatically generate multiple distinct and natural hand grasps, given specific contact region of an object in 3D? This motivates us to consider a novel task of \textit{Region Controllable Hand Grasp Generation (RegionGrasp)}, as follows: given as input a 3D object, together with its specific surface area selected as the intended contact region, to generate a diverse set of plausible hand grasps of the object, where the thumb finger tip touches the object surface on the contact region. To address this task, RegionGrasp-CVAE is proposed, which consists of two main parts. First, to enable contact region-awareness, we propose ConditionNet as the condition encoder that includes in it a transformer-backboned object encoder, O-Enc; a pretraining strategy is adopted by O-Enc, where the point patches of object surface are randomly masked off and subsequently restored, to further capture surface geometric information of the object. Second, to realize interaction awareness, HOINet is introduced to encode hand-object interaction features by entangling high-level hand features with embedded object features through geometric-aware multi-head cross attention. Empirical evaluations demonstrate the effectiveness of our approach qualitatively and quantitatively where it is shown to compare favorably with respect to the state of the art methods.