TriHorn-Net: A Model for Accurate Depth-Based 3D Hand Pose Estimation

TL;DR

TriHorn-Net introduces a novel approach for 3D hand pose estimation from depth images by decomposing the problem and employing a new data augmentation method, significantly improving accuracy over existing methods.

Contribution

The paper presents TriHorn-Net, a new model that decomposes 3D hand pose estimation and introduces PixDropout, a novel data augmentation technique for depth images.

Findings

Outperforms state-of-the-art methods on three benchmark datasets.

Effective decomposition of pose estimation into UV and depth components.

PixDropout enhances model robustness and accuracy.

Abstract

3D hand pose estimation methods have made significant progress recently. However, the estimation accuracy is often far from sufficient for specific real-world applications, and thus there is significant room for improvement. This paper proposes TriHorn-Net, a novel model that uses specific innovations to improve hand pose estimation accuracy on depth images. The first innovation is the decomposition of the 3D hand pose estimation into the estimation of 2D joint locations in the depth image space (UV), and the estimation of their corresponding depths aided by two complementary attention maps. This decomposition prevents depth estimation, which is a more difficult task, from interfering with the UV estimations at both the prediction and feature levels. The second innovation is PixDropout, which is, to the best of our knowledge, the first appearance-based data augmentation method for hand depth images. Experimental results demonstrate that the proposed model outperforms the state-of-the-art methods on three public benchmark datasets. Our implementation is available at https://github.com/mrezaei92/TriHorn-Net.