HINT: Learning Complete Human Neural Representations from Limited Viewpoints

TL;DR

HINT is a NeRF-based method that reconstructs detailed, complete human models from limited viewpoints by leveraging symmetry, explicit 3D supervision, and large datasets, enabling more accurate avatar generation.

Contribution

The paper introduces HINT, a novel NeRF-based approach that learns full human representations from few views using symmetry priors and multi-source supervision.

Findings

Achieves over 15% PSNR improvement over previous methods.

Successfully reconstructs complete human models from limited viewpoints.

Utilizes symmetry and explicit 3D supervision for enhanced detail.

Abstract

No augmented application is possible without animated humanoid avatars. At the same time, generating human replicas from real-world monocular hand-held or robotic sensor setups is challenging due to the limited availability of views. Previous work showed the feasibility of virtual avatars but required the presence of 360 degree views of the targeted subject. To address this issue, we propose HINT, a NeRF-based algorithm able to learn a detailed and complete human model from limited viewing angles. We achieve this by introducing a symmetry prior, regularization constraints, and training cues from large human datasets. In particular, we introduce a sagittal plane symmetry prior to the appearance of the human, directly supervise the density function of the human model using explicit 3D body modeling, and leverage a co-learned human digitization network as additional supervision for the unseen angles. As a result, our method can reconstruct complete humans even from a few viewing angles, increasing performance by more than 15% PSNR compared to previous state-of-the-art algorithms.