Efficient Virtual View Selection for 3D Hand Pose Estimation

TL;DR

This paper introduces a virtual view selection and fusion module that automatically chooses effective viewpoints for 3D hand pose estimation from single depth images, improving accuracy and robustness across multiple datasets.

Contribution

It presents a novel virtual view selection and fusion approach using a lightweight network, enhancing 3D hand pose estimation performance over existing methods.

Findings

Outperforms state-of-the-art on NYU and ICVL datasets.

Achieves competitive results on Hands2019-Task1.

Effective virtual view selection improves robustness and accuracy.

Abstract

3D hand pose estimation from single depth is a fundamental problem in computer vision, and has wide applications.However, the existing methods still can not achieve satisfactory hand pose estimation results due to view variation and occlusion of human hand. In this paper, we propose a new virtual view selection and fusion module for 3D hand pose estimation from single depth.We propose to automatically select multiple virtual viewpoints for pose estimation and fuse the results of all and find this empirically delivers accurate and robust pose estimation. In order to select most effective virtual views for pose fusion, we evaluate the virtual views based on the confidence of virtual views using a light-weight network via network distillation. Experiments on three main benchmark datasets including NYU, ICVL and Hands2019 demonstrate that our method outperforms the state-of-the-arts on NYU and ICVL, and achieves very competitive performance on Hands2019-Task1, and our proposed virtual view selection and fusion module is both effective for 3D hand pose estimation.