Local and Global Point Cloud Reconstruction for 3D Hand Pose Estimation

TL;DR

This paper introduces a new method for 3D hand pose estimation from a single RGB image by reconstructing detailed point clouds using a novel pipeline and a new dataset, outperforming existing methods.

Contribution

A novel pipeline for local and global point cloud reconstruction of hands from RGB images, incorporating a new multi-view hand dataset and a latent pose representation.

Findings

Outperforms existing methods in 3D pose estimation

Reconstructs realistic, complete 3D hand point clouds

Validates effectiveness on multiple datasets

Abstract

This paper addresses the 3D point cloud reconstruction and 3D pose estimation of the human hand from a single RGB image. To that end, we present a novel pipeline for local and global point cloud reconstruction using a 3D hand template while learning a latent representation for pose estimation. To demonstrate our method, we introduce a new multi-view hand posture dataset to obtain complete 3D point clouds of the hand in the real world. Experiments on our newly proposed dataset and four public benchmarks demonstrate the model's strengths. Our method outperforms competitors in 3D pose estimation while reconstructing realistic-looking complete 3D hand point clouds.