InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image

TL;DR

This paper introduces a large-scale dataset and a baseline network for 3D interacting hand pose estimation from a single RGB image, addressing the gap in existing research focused mainly on single hand pose estimation.

Contribution

The paper presents InterHand2.6M, a new dataset with 2.6 million labeled frames of single and interacting hands, and InterNet, a baseline network for 3D interacting hand pose estimation.

Findings

Significant accuracy improvements in 3D interacting hand pose estimation using the new dataset.

InterNet achieves strong baseline performance on InterHand2.6M.

Demonstrates feasibility of 3D interacting hand pose estimation from general images.

Abstract

Analysis of hand-hand interactions is a crucial step towards better understanding human behavior. However, most researches in 3D hand pose estimation have focused on the isolated single hand case. Therefore, we firstly propose (1) a large-scale dataset, InterHand2.6M, and (2) a baseline network, InterNet, for 3D interacting hand pose estimation from a single RGB image. The proposed InterHand2.6M consists of \textbf{2.6M labeled single and interacting hand frames} under various poses from multiple subjects. Our InterNet simultaneously performs 3D single and interacting hand pose estimation. In our experiments, we demonstrate big gains in 3D interacting hand pose estimation accuracy when leveraging the interacting hand data in InterHand2.6M. We also report the accuracy of InterNet on InterHand2.6M, which serves as a strong baseline for this new dataset. Finally, we show 3D interacting hand pose estimation results from general images. Our code and dataset are available at https://mks0601.github.io/InterHand2.6M/.