Hybrid One-Shot 3D Hand Pose Estimation by Exploiting Uncertainties

TL;DR

This paper introduces a hybrid 3D hand pose estimation method from a single depth image that combines data-driven hypotheses with model-based refinement, achieving high accuracy and anatomical validity without manual initialization.

Contribution

It proposes a novel hybrid approach that exploits uncertainty in joint proposals to improve 3D hand pose estimation from depth images.

Findings

Outperforms state-of-the-art methods on standard datasets.

Provides anatomically valid and accurate hand poses.

Does not require manual initialization or suffer from tracking failures.

Abstract

Model-based approaches to 3D hand tracking have been shown to perform well in a wide range of scenarios. However, they require initialisation and cannot recover easily from tracking failures that occur due to fast hand motions. Data-driven approaches, on the other hand, can quickly deliver a solution, but the results often suffer from lower accuracy or missing anatomical validity compared to those obtained from model-based approaches. In this work we propose a hybrid approach for hand pose estimation from a single depth image. First, a learned regressor is employed to deliver multiple initial hypotheses for the 3D position of each hand joint. Subsequently, the kinematic parameters of a 3D hand model are found by deliberately exploiting the inherent uncertainty of the inferred joint proposals. This way, the method provides anatomically valid and accurate solutions without requiring manual initialisation or suffering from track losses. Quantitative results on several standard datasets demonstrate that the proposed method outperforms state-of-the-art representatives of the model-based, data-driven and hybrid paradigms.