Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation

TL;DR

This paper introduces a lightweight, recursive neural network for 3D hand pose estimation that adaptively refines predictions through learned gating, achieving superior accuracy and efficiency over existing methods.

Contribution

It proposes a novel recursive refinement framework with learned gating criteria for efficient 3D hand pose estimation, enhancing accuracy and computational efficiency.

Findings

Outperforms state-of-the-art methods in accuracy.

Reduces computational cost through recursive refinement.

Effectively estimates uncertainty for adaptive iteration control.

Abstract

While hand pose estimation is a critical component of most interactive extended reality and gesture recognition systems, contemporary approaches are not optimized for computational and memory efficiency. In this paper, we propose a tiny deep neural network of which partial layers are recursively exploited for refining its previous estimations. During its iterative refinements, we employ learned gating criteria to decide whether to exit from the weight-sharing loop, allowing per-sample adaptation in our model. Our network is trained to be aware of the uncertainty in its current predictions to efficiently gate at each iteration, estimating variances after each loop for its keypoint estimates. Additionally, we investigate the effectiveness of end-to-end and progressive training protocols for our recursive structure on maximizing the model capacity. With the proposed setting, our method consistently outperforms state-of-the-art 2D/3D hand pose estimation approaches in terms of both accuracy and efficiency for widely used benchmarks.