Model-based Deep Hand Pose Estimation

TL;DR

This paper introduces a novel deep learning method for hand pose estimation that integrates a forward kinematics layer to ensure geometric validity, eliminating the need for separate post-processing steps.

Contribution

It presents the first feasible integration of a non-linear generative hand model within deep learning for pose estimation, achieving state-of-the-art results.

Findings

Achieves state-of-the-art performance on public datasets.

Ensures geometric validity of hand poses within the neural network.

Eliminates the need for separate model fitting steps.

Abstract

Previous learning based hand pose estimation methods does not fully exploit the prior information in hand model geometry. Instead, they usually rely a separate model fitting step to generate valid hand poses. Such a post processing is inconvenient and sub-optimal. In this work, we propose a model based deep learning approach that adopts a forward kinematics based layer to ensure the geometric validity of estimated poses. For the first time, we show that embedding such a non-linear generative process in deep learning is feasible for hand pose estimation. Our approach is verified on challenging public datasets and achieves state-of-the-art performance.