HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texture

TL;DR

HiFiHR introduces a novel single-image 3D hand reconstruction method that produces high-fidelity textures and accurate hand meshes by combining render-and-compare techniques with a parametric hand model, outperforming existing methods.

Contribution

The paper presents a new approach that enhances texture quality and pose accuracy in 3D hand reconstruction from a single image using a render-and-compare framework with various supervision levels.

Findings

Outperforms state-of-the-art in texture reconstruction quality.

Maintains comparable pose and shape estimation accuracy.

Effective across different supervision settings.

Abstract

We present HiFiHR, a high-fidelity hand reconstruction approach that utilizes render-and-compare in the learning-based framework from a single image, capable of generating visually plausible and accurate 3D hand meshes while recovering realistic textures. Our method achieves superior texture reconstruction by employing a parametric hand model with predefined texture assets, and by establishing a texture reconstruction consistency between the rendered and input images during training. Moreover, based on pretraining the network on an annotated dataset, we apply varying degrees of supervision using our pipeline, i.e., self-supervision, weak supervision, and full supervision, and discuss the various levels of contributions of the learned high-fidelity textures in enhancing hand pose and shape estimation. Experimental results on public benchmarks including FreiHAND and HO-3D demonstrate that our method outperforms the state-of-the-art hand reconstruction methods in texture reconstruction quality while maintaining comparable accuracy in pose and shape estimation. Our code is available at https://github.com/viridityzhu/HiFiHR.