NOFA: NeRF-based One-shot Facial Avatar Reconstruction

TL;DR

This paper introduces NOFA, a novel one-shot neural radiance field framework that reconstructs high-quality 3D facial avatars from a single image, enabling detailed control over expressions and outperforming existing methods.

Contribution

The work presents a new one-shot NeRF-based facial avatar reconstruction method using a 3D GAN prior and a deformation field for expression control, addressing generalization and multi-view limitations.

Findings

Achieves superior synthesis quality compared to state-of-the-art methods.

Reconstructs high-fidelity 3D facial avatars from a single image.

Enables fine-grained facial expression control.

Abstract

3D facial avatar reconstruction has been a significant research topic in computer graphics and computer vision, where photo-realistic rendering and flexible controls over poses and expressions are necessary for many related applications. Recently, its performance has been greatly improved with the development of neural radiance fields (NeRF). However, most existing NeRF-based facial avatars focus on subject-specific reconstruction and reenactment, requiring multi-shot images containing different views of the specific subject for training, and the learned model cannot generalize to new identities, limiting its further applications. In this work, we propose a one-shot 3D facial avatar reconstruction framework that only requires a single source image to reconstruct a high-fidelity 3D facial avatar. For the challenges of lacking generalization ability and missing multi-view information, we leverage the generative prior of 3D GAN and develop an efficient encoder-decoder network to reconstruct the canonical neural volume of the source image, and further propose a compensation network to complement facial details. To enable fine-grained control over facial dynamics, we propose a deformation field to warp the canonical volume into driven expressions. Through extensive experimental comparisons, we achieve superior synthesis results compared to several state-of-the-art methods.