PCIE_EgoHandPose Solution for EgoExo4D Hand Pose Challenge

TL;DR

This paper introduces the PCIE_EgoHandPose solution utilizing a novel Vision Transformer architecture to accurately estimate 3D hand poses from egocentric RGB videos, achieving top performance in the CVPR2024 challenge.

Contribution

The paper presents the Hand Pose Vision Transformer (HP-ViT), a new transformer-based model specifically designed for 3D hand pose estimation from egocentric videos, and demonstrates its effectiveness in a competitive challenge.

Findings

Achieved 1st place in the CVPR2024 EgoExo4D Hand Pose Challenge.

Attained 25.51 MPJPE and 8.49 PA-MPJPE scores.

Proposed a transformer-based architecture for hand pose estimation.

Abstract

This report presents our team's 'PCIE_EgoHandPose' solution for the EgoExo4D Hand Pose Challenge at CVPR2024. The main goal of the challenge is to accurately estimate hand poses, which involve 21 3D joints, using an RGB egocentric video image provided for the task. This task is particularly challenging due to the subtle movements and occlusions. To handle the complexity of the task, we propose the Hand Pose Vision Transformer (HP-ViT). The HP-ViT comprises a ViT backbone and transformer head to estimate joint positions in 3D, utilizing MPJPE and RLE loss function. Our approach achieved the 1st position in the Hand Pose challenge with 25.51 MPJPE and 8.49 PA-MPJPE. Code is available at https://github.com/KanokphanL/PCIE_EgoHandPose