Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation

TL;DR

This paper introduces a generative model-based loss function for depth-based hand pose estimation that enables training with limited supervision and robustness to annotation errors, improving accuracy over traditional methods.

Contribution

It presents a novel partially-supervised training approach using a generative loss that handles annotation errors and reduces annotation requirements.

Findings

Achieves comparable results to fully-supervised methods with limited supervision.

Effectively trains on datasets with erroneous annotations.

Produces more accurate depth image explanations than noisy ground truth.

Abstract

We propose to use a model-based generative loss for training hand pose estimators on depth images based on a volumetric hand model. This additional loss allows training of a hand pose estimator that accurately infers the entire set of 21 hand keypoints while only using supervision for 6 easy-to-annotate keypoints (fingertips and wrist). We show that our partially-supervised method achieves results that are comparable to those of fully-supervised methods which enforce articulation consistency. Moreover, for the first time we demonstrate that such an approach can be used to train on datasets that have erroneous annotations, i.e. "ground truth" with notable measurement errors, while obtaining predictions that explain the depth images better than the given "ground truth".