1st Place Solution of Multiview Egocentric Hand Tracking Challenge ECCV2024

TL;DR

This paper presents a novel multi-view egocentric hand tracking method that combines multi-view images, camera parameters, data augmentation, and neural smoothing to improve accuracy in VR applications.

Contribution

The authors introduce a new approach that leverages multi-view inputs, augmentation techniques, and neural post-processing for enhanced hand tracking accuracy.

Findings

Achieved 13.92mm MPJPE on Umetrack dataset

Achieved 21.66mm MPJPE on HOT3D dataset

Improved hand pose estimation accuracy with neural smoothing

Abstract

Multi-view egocentric hand tracking is a challenging task and plays a critical role in VR interaction. In this report, we present a method that uses multi-view input images and camera extrinsic parameters to estimate both hand shape and pose. To reduce overfitting to the camera layout, we apply crop jittering and extrinsic parameter noise augmentation. Additionally, we propose an offline neural smoothing post-processing method to further improve the accuracy of hand position and pose. Our method achieves 13.92mm MPJPE on the Umetrack dataset and 21.66mm MPJPE on the HOT3D dataset.