HGGT: Robust and Flexible 3D Hand Mesh Reconstruction from Uncalibrated Images

TL;DR

This paper introduces HGGT, a novel method for 3D hand mesh reconstruction from uncalibrated images that combines the advantages of single-view and multi-view approaches, enabling accurate and flexible deployment in real-world scenarios.

Contribution

We propose a feed-forward architecture that jointly infers 3D hand meshes and camera poses from uncalibrated views, bridging the gap between single-view and multi-view methods.

Findings

Outperforms state-of-the-art benchmarks in 3D hand reconstruction.

Demonstrates strong generalization to uncalibrated, in-the-wild scenarios.

Enables deployment on consumer-grade RGB cameras without calibration.

Abstract

Recovering high-fidelity 3D hand geometry from images is a critical task in computer vision, holding significant value for domains such as robotics, animation and VR/AR. Crucially, scalable applications demand both accuracy and deployment flexibility, requiring the ability to leverage massive amounts of unstructured image data from the internet or enable deployment on consumer-grade RGB cameras without complex calibration. However, current methods face a dilemma. While single-view approaches are easy to deploy, they suffer from depth ambiguity and occlusion. Conversely, multi-view systems resolve these uncertainties but typically demand fixed, calibrated setups, limiting their real-world utility. To bridge this gap, we draw inspiration from 3D foundation models that learn explicit geometry directly from visual data. By reformulating hand reconstruction from arbitrary views as a visual-geometry grounded task, we propose a feed-forward architecture that, for the first time in literature, jointly infers 3D hand meshes and camera poses from uncalibrated views. Extensive evaluations show that our approach outperforms state-of-the-art benchmarks and demonstrates strong generalization to uncalibrated, in-the-wild scenarios. Here is the link of our project page: https://lym29.github.io/HGGT/.