Portrait Neural Radiance Fields from a Single Image

TL;DR

This paper introduces a novel method to generate Neural Radiance Fields (NeRF) from a single portrait image by leveraging meta-learning and 3D face models, enabling high-quality view synthesis without multiple images.

Contribution

It proposes a meta-learning approach to estimate NeRF from a single portrait, using face models for better generalization to unseen faces.

Findings

Effective single-image NeRF estimation demonstrated on real portraits

Outperforms existing methods in view synthesis quality

Generalizes well to unseen faces in real-world images

Abstract

We present a method for estimating Neural Radiance Fields (NeRF) from a single headshot portrait. While NeRF has demonstrated high-quality view synthesis, it requires multiple images of static scenes and thus impractical for casual captures and moving subjects. In this work, we propose to pretrain the weights of a multilayer perceptron (MLP), which implicitly models the volumetric density and colors, with a meta-learning framework using a light stage portrait dataset. To improve the generalization to unseen faces, we train the MLP in the canonical coordinate space approximated by 3D face morphable models. We quantitatively evaluate the method using controlled captures and demonstrate the generalization to real portrait images, showing favorable results against state-of-the-arts.