Towards Good Practices for Deep 3D Hand Pose Estimation

TL;DR

This paper introduces a tree-structured Region Ensemble Network (REN) for 3D hand pose estimation from depth images, significantly improving accuracy over existing deep learning methods by leveraging ensemble strategies and training techniques.

Contribution

The paper proposes a novel REN architecture with grid partitioning and ensemble learning for improved 3D hand pose regression, outperforming prior methods.

Findings

Achieves state-of-the-art results on three public hand pose datasets.

Improves hand joint localization accuracy with training strategies like data augmentation.

Demonstrates effectiveness on fingertip detection and human pose datasets.

Abstract

3D hand pose estimation from single depth image is an important and challenging problem for human-computer interaction. Recently deep convolutional networks (ConvNet) with sophisticated design have been employed to address it, but the improvement over traditional random forest based methods is not so apparent. To exploit the good practice and promote the performance for hand pose estimation, we propose a tree-structured Region Ensemble Network (REN) for directly 3D coordinate regression. It first partitions the last convolution outputs of ConvNet into several grid regions. The results from separate fully-connected (FC) regressors on each regions are then integrated by another FC layer to perform the estimation. By exploitation of several training strategies including data augmentation and smooth $L_1$ loss, proposed REN can significantly improve the performance of ConvNet to localize hand joints. The experimental results demonstrate that our approach achieves the best performance among state-of-the-art algorithms on three public hand pose datasets. We also experiment our methods on fingertip detection and human pose datasets and obtain state-of-the-art accuracy.