High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decomposition

TL;DR

This paper introduces a scalable frequency decomposition network for high-fidelity 3D hand reconstruction from a single image, capturing detailed personalized hand mesh features by leveraging frequency domain analysis and a novel loss function.

Contribution

It proposes a frequency split network with a coarse-to-fine scheme and a new frequency decomposition loss, enabling detailed and scalable 3D hand mesh reconstruction from single images.

Findings

Achieves detailed 3D hand reconstructions with preserved high-frequency features.

Introduces MSNR, a new metric for evaluating mesh detail quality.

Demonstrates superior performance over existing methods in detail recovery.

Abstract

Despite the impressive performance obtained by recent single-image hand modeling techniques, they lack the capability to capture sufficient details of the 3D hand mesh. This deficiency greatly limits their applications when high-fidelity hand modeling is required, e.g., personalized hand modeling. To address this problem, we design a frequency split network to generate 3D hand mesh using different frequency bands in a coarse-to-fine manner. To capture high-frequency personalized details, we transform the 3D mesh into the frequency domain, and propose a novel frequency decomposition loss to supervise each frequency component. By leveraging such a coarse-to-fine scheme, hand details that correspond to the higher frequency domain can be preserved. In addition, the proposed network is scalable, and can stop the inference at any resolution level to accommodate different hardware with varying computational powers. To quantitatively evaluate the performance of our method in terms of recovering personalized shape details, we introduce a new evaluation metric named Mean Signal-to-Noise Ratio (MSNR) to measure the signal-to-noise ratio of each mesh frequency component. Extensive experiments demonstrate that our approach generates fine-grained details for high-fidelity 3D hand reconstruction, and our evaluation metric is more effective for measuring mesh details compared with traditional metrics.