3D Hand Pose Estimation in Everyday Egocentric Images
Aditya Prakash, Ruisen Tu, Matthew Chang, Saurabh Gupta

TL;DR
This paper investigates the challenges of 3D hand pose estimation in egocentric images, analyzing factors like perspective distortion and lack of annotations, and introduces WildHands, a system that outperforms existing methods across multiple datasets.
Contribution
The paper systematically studies the impact of various practices on 3D hand pose estimation in the wild and presents WildHands, a new system that achieves state-of-the-art results with less data and smaller size.
Findings
WildHands outperforms previous methods by 7.4% to 66% on diverse datasets.
Incorporating camera info and auxiliary supervision improves estimation accuracy.
WildHands is 10x smaller and trained on 5x less data than comparable systems.
Abstract
3D hand pose estimation in everyday egocentric images is challenging for several reasons: poor visual signal (occlusion from the object of interaction, low resolution & motion blur), large perspective distortion (hands are close to the camera), and lack of 3D annotations outside of controlled settings. While existing methods often use hand crops as input to focus on fine-grained visual information to deal with poor visual signal, the challenges arising from perspective distortion and lack of 3D annotations in the wild have not been systematically studied. We focus on this gap and explore the impact of different practices, i.e. crops as input, incorporating camera information, auxiliary supervision, scaling up datasets. We provide several insights that are applicable to both convolutional and transformer models leading to better performance. Based on our findings, we also present…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSmart Agriculture and AI
MethodsFocus
