Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions

TL;DR

This paper introduces Multi-NeuS, a neural implicit function-based method that reconstructs textured 3D human head models from minimal input views, learning priors from data without needing 3D scans.

Contribution

It extends NeuS to handle multiple human head objects simultaneously, enabling few-shot and one-shot 3D head reconstruction from limited video data.

Findings

Effective 3D head reconstruction from single or few images.

Learns priors of human heads without 3D scans.

Generalizes to unseen head shapes.

Abstract

We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation $\unicode{x2013}$ neural implicit functions $\unicode{x2013}$ which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.