HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field

TL;DR

This paper introduces HAvatar, a hybrid explicit-implicit 3D head avatar model that combines neural radiance fields with parametric templates, enabling high-fidelity, view-consistent, and animatable head avatars with improved realism and expression control.

Contribution

The paper presents a novel Facial Model Conditioned Neural Radiance Field that integrates prior information into an implicit field, enhancing realism, expression accuracy, and animation stability in 3D head avatars.

Findings

Achieves state-of-the-art performance in 3D head avatar animation.

Produces high-resolution, realistic, and view-consistent head synthesis.

Effectively overcomes shape inconsistency and improves animation stability.

Abstract

The problem of modeling an animatable 3D human head avatar under light-weight setups is of significant importance but has not been well solved. Existing 3D representations either perform well in the realism of portrait images synthesis or the accuracy of expression control, but not both. To address the problem, we introduce a novel hybrid explicit-implicit 3D representation, Facial Model Conditioned Neural Radiance Field, which integrates the expressiveness of NeRF and the prior information from the parametric template. At the core of our representation, a synthetic-renderings-based condition method is proposed to fuse the prior information from the parametric model into the implicit field without constraining its topological flexibility. Besides, based on the hybrid representation, we properly overcome the inconsistent shape issue presented in existing methods and improve the animation stability. Moreover, by adopting an overall GAN-based architecture using an image-to-image translation network, we achieve high-resolution, realistic and view-consistent synthesis of dynamic head appearance. Experiments demonstrate that our method can achieve state-of-the-art performance for 3D head avatar animation compared with previous methods.