Identity-Aware Hand Mesh Estimation and Personalization from RGB Images
Deying Kong, Linguang Zhang, Liangjian Chen, Haoyu Ma, Xiangyi Yan,, Shanlin Sun, Xingwei Liu, Kun Han, Xiaohui Xie

TL;DR
This paper introduces an identity-aware approach for 3D hand mesh reconstruction from RGB images, leveraging intrinsic shape parameters for personalization and improved accuracy in AR/VR applications.
Contribution
It presents a novel identity-aware hand mesh estimation model and a personalization pipeline for unseen subjects using minimal unlabeled images.
Findings
Outperforms existing anonymous methods on public datasets.
Personalization effectively calibrates shape parameters for new users.
Identity information improves hand mesh reconstruction accuracy.
Abstract
Reconstructing 3D hand meshes from monocular RGB images has attracted increasing amount of attention due to its enormous potential applications in the field of AR/VR. Most state-of-the-art methods attempt to tackle this task in an anonymous manner. Specifically, the identity of the subject is ignored even though it is practically available in real applications where the user is unchanged in a continuous recording session. In this paper, we propose an identity-aware hand mesh estimation model, which can incorporate the identity information represented by the intrinsic shape parameters of the subject. We demonstrate the importance of the identity information by comparing the proposed identity-aware model to a baseline which treats subject anonymously. Furthermore, to handle the use case where the test subject is unseen, we propose a novel personalization pipeline to calibrate the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Hand Gesture Recognition Systems · Face recognition and analysis
MethodsTest
