Adaptive Wasserstein Hourglass for Weakly Supervised Hand Pose Estimation from Monocular RGB

TL;DR

This paper introduces a novel domain adaptation method called Adaptive Wasserstein Hourglass that improves weakly-supervised 3D hand pose estimation from monocular RGB images by bridging the gap between synthetic and real datasets.

Contribution

It proposes a domain adaptation framework that learns common hand structure features to enhance 3D pose estimation from weakly labeled real-world data.

Findings

Improved accuracy in 3D hand pose estimation from monocular RGB images.

Effective domain-invariant feature learning between synthetic and real datasets.

Enhanced generalization to real-world scenarios.

Abstract

Insufficient labeled training datasets is one of the bottlenecks of 3D hand pose estimation from monocular RGB images. Synthetic datasets have a large number of images with precise annotations, but the obvious difference with real-world datasets impacts the generalization. Little work has been done to bridge the gap between two domains over their wide difference. In this paper, we propose a domain adaptation method called Adaptive Wasserstein Hourglass (AW Hourglass) for weakly-supervised 3D hand pose estimation, which aims to distinguish the difference and explore the common characteristics (e.g. hand structure) of synthetic and real-world datasets. Learning the common characteristics helps the network focus on pose-related information. The similarity of the characteristics makes it easier to enforce domain-invariant constraints. During training, based on the relation between these common characteristics and 3D pose learned from fully-annotated synthetic datasets, it is beneficial for the network to restore the 3D pose of weakly labeled real-world datasets with the aid of 2D annotations and depth images. While in testing, the network predicts the 3D pose with the input of RGB.