GeoHand: Unlocking Prior Geometry Knowledge for Monocular 3D Hand Reconstruction

TL;DR

GeoHand leverages prior geometric knowledge and a novel fusion strategy to significantly improve monocular 3D hand reconstruction, especially in occluded and complex scenes.

Contribution

The paper introduces GeoHand, a framework that extracts high-quality geometric priors from a pretrained estimator and adapts them for detailed hand reconstruction using a novel fusion and refinement strategy.

Findings

Achieves state-of-the-art results on FreiHAND, DexYCB, and HO3Dv3 datasets.

Excels in scenarios with severe occlusions and hand-object interactions.

Effectively integrates geometric priors with RGB features for precise reconstruction.

Abstract

Monocular 3D hand reconstruction is intrinsically a geometric problem, yet RGB appearance features alone often struggle to resolve severe ambiguities caused by self-occlusions and hand-object interactions. While introducing depth can explicitly provide spatial cues, raw sensor-captured depth maps are extensively noisy and incomplete, limiting their usefulness for fine-grained hand reconstruction. To bridge this gap, we propose GeoHand, a novel framework that unlocks high-quality geometric priors from a frozen foundational monocular geometry estimator (MoGe2). Recognizing that these priors are oriented toward general scenes, we introduce a map-level GeoAdapter to recalibrate the spatial features, specifically adapting them for detailed hand reconstruction. Furthermore, to systematically integrate these adapted priors without overwhelming intrinsic RGB appearance cues, we employ a gated cross-modal token fusion strategy. Finally, to secure precise local articulation, we design a Keypoint-Queried Iterative Refiner (KQIR) that uses projected joint locations to query geometry-aware image features for spatial correction. By combining global geometric disambiguation with local refinement in a unified pipeline, GeoHand achieves state-of-the-art performance on FreiHAND, DexYCB, and HO3Dv3, especially under severe occlusions and hand-object interactions.