3D Hand Pose Estimation via Regularized Graph Representation Learning

TL;DR

This paper introduces a novel graph-based learning framework with structural regularization and adversarial training to improve 3D hand pose estimation from monocular RGB images, achieving state-of-the-art results.

Contribution

It proposes a regularized graph representation learning method with a prior hand model and adversarial framework to better capture hand joint dependencies.

Findings

Achieves new state-of-the-art performance on five benchmarks.

Effectively models hand joint structure with residual graph convolution.

Utilizes bone-constrained loss functions for structural accuracy.

Abstract

This paper addresses the problem of 3D hand pose estimation from a monocular RGB image. While previous methods have shown great success, the structure of hands has not been fully exploited, which is critical in pose estimation. To this end, we propose a regularized graph representation learning under a conditional adversarial learning framework for 3D hand pose estimation, aiming to capture structural inter-dependencies of hand joints. In particular, we estimate an initial hand pose from a parametric hand model as a prior of hand structure, which regularizes the inference of the structural deformation in the prior pose for accurate graph representation learning via residual graph convolution. To optimize the hand structure further, we propose two bone-constrained loss functions, which characterize the morphable structure of hand poses explicitly. Also, we introduce an adversarial learning framework conditioned on the input image with a multi-source discriminator, which imposes the structural constraints onto the distribution of generated 3D hand poses for anthropomorphically valid hand poses. Extensive experiments demonstrate that our model sets the new state-of-the-art in 3D hand pose estimation from a monocular image on five standard benchmarks.