DIREG3D: DIrectly REGress 3D Hands from Multiple Cameras

TL;DR

DIREG3D introduces a comprehensive framework for accurate 3D hand tracking from monocular and multi-view cameras by leveraging geometric and visual cues for precise hand mesh parameter regression.

Contribution

It presents a novel method that integrates camera parameters, 2D cues, and multi-view data to improve 3D hand pose estimation accuracy.

Findings

Effective monocular 3D hand pose estimation using 2D cues and camera info.

Multi-view fusion enhances 3D hand pose accuracy.

Utilization of hand size, distance, and distortion improves results.

Abstract

In this paper, we present DIREG3D, a holistic framework for 3D Hand Tracking. The proposed framework is capable of utilizing camera intrinsic parameters, 3D geometry, intermediate 2D cues, and visual information to regress parameters for accurately representing a Hand Mesh model. Our experiments show that information like the size of the 2D hand, its distance from the optical center, and radial distortion is useful for deriving highly reliable 3D poses in camera space from just monocular information. Furthermore, we extend these results to a multi-view camera setup by fusing features from different viewpoints.