Hand3D: Hand Pose Estimation using 3D Neural Network

TL;DR

This paper introduces a 3D neural network architecture that directly estimates 3D hand poses from depth images by converting them into volumetric data, eliminating the need for intermediate processing or reference points.

Contribution

It presents a novel 3D CNN approach that directly predicts 3D hand poses from volumetric depth data, integrating local and global features without requiring initialization.

Findings

Achieves state-of-the-art performance on public benchmarks

Effectively uses synthetic data to improve pose coverage

No need for ground truth reference points for initialization

Abstract

We propose a novel 3D neural network architecture for 3D hand pose estimation from a single depth image. Different from previous works that mostly run on 2D depth image domain and require intermediate or post process to bring in the supervision from 3D space, we convert the depth map to a 3D volumetric representation, and feed it into a 3D convolutional neural network(CNN) to directly produce the pose in 3D requiring no further process. Our system does not require the ground truth reference point for initialization, and our network architecture naturally integrates both local feature and global context in 3D space. To increase the coverage of the hand pose space of the training data, we render synthetic depth image by transferring hand pose from existing real image datasets. We evaluation our algorithm on two public benchmarks and achieve the state-of-the-art performance. The synthetic hand pose dataset will be available.