HandDAGT: A Denoising Adaptive Graph Transformer for 3D Hand Pose Estimation

TL;DR

HandDAGT introduces a novel transformer-based approach with a denoising training strategy to improve 3D hand pose estimation accuracy and robustness under occlusion conditions, outperforming existing methods.

Contribution

This paper presents HandDAGT, a new graph transformer model with an adaptive attention mechanism and denoising training for robust 3D hand pose estimation.

Findings

Significantly outperforms existing methods on four benchmark datasets.

Effectively handles self-occlusion and object interaction scenarios.

Demonstrates robustness and accuracy improvements in challenging conditions.

Abstract

The extraction of keypoint positions from input hand frames, known as 3D hand pose estimation, is crucial for various human-computer interaction applications. However, current approaches often struggle with the dynamic nature of self-occlusion of hands and intra-occlusion with interacting objects. To address this challenge, this paper proposes the Denoising Adaptive Graph Transformer, HandDAGT, for hand pose estimation. The proposed HandDAGT leverages a transformer structure to thoroughly explore effective geometric features from input patches. Additionally, it incorporates a novel attention mechanism to adaptively weigh the contribution of kinematic correspondence and local geometric features for the estimation of specific keypoints. This attribute enables the model to adaptively employ kinematic and local information based on the occlusion situation, enhancing its robustness and accuracy. Furthermore, we introduce a novel denoising training strategy aimed at improving the model's robust performance in the face of occlusion challenges. Experimental results show that the proposed model significantly outperforms the existing methods on four challenging hand pose benchmark datasets. Codes and pre-trained models are publicly available at https://github.com/cwc1260/HandDAGT.