Hand-object reconstruction via interaction-aware graph attention mechanism

TL;DR

This paper introduces an interaction-aware graph attention mechanism for hand-object pose estimation, improving the understanding of their interactions and enhancing physical plausibility in reconstructions.

Contribution

It proposes a novel graph-based refinement method that fully exploits intra- and inter-graph connections to better model hand-object interactions.

Findings

Improved physical plausibility in hand-object reconstructions

Enhanced accuracy over existing graph neural network approaches

Effective modeling of hand-object contact and interaction

Abstract

Estimating the poses of both a hand and an object has become an important area of research due to the growing need for advanced vision computing. The primary challenge involves understanding and reconstructing how hands and objects interact, such as contact and physical plausibility. Existing approaches often adopt a graph neural network to incorporate spatial information of hand and object meshes. However, these approaches have not fully exploited the potential of graphs without modification of edges within and between hand- and object-graphs. We propose a graph-based refinement method that incorporates an interaction-aware graph-attention mechanism to account for hand-object interactions. Using edges, we establish connections among closely correlated nodes, both within individual graphs and across different graphs. Experiments demonstrate the effectiveness of our proposed method with notable improvements in the realm of physical plausibility.