ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations

TL;DR

ImFace introduces a nonlinear, continuous 3D face model using implicit neural representations, enabling more accurate and detailed face reconstructions with disentangled identity and expression modeling.

Contribution

It proposes a novel nonlinear 3D morphable face model with implicit neural representations and a preprocessing pipeline for common facial surfaces.

Findings

Outperforms existing models in accuracy and detail.

Effectively disentangles identity and expression.

Works with non-watertight facial data.

Abstract

Precise representations of 3D faces are beneficial to various computer vision and graphics applications. Due to the data discretization and model linearity, however, it remains challenging to capture accurate identity and expression clues in current studies. This paper presents a novel 3D morphable face model, namely ImFace, to learn a nonlinear and continuous space with implicit neural representations. It builds two explicitly disentangled deformation fields to model complex shapes associated with identities and expressions, respectively, and designs an improved learning strategy to extend embeddings of expressions to allow more diverse changes. We further introduce a Neural Blend-Field to learn sophisticated details by adaptively blending a series of local fields. In addition to ImFace, an effective preprocessing pipeline is proposed to address the issue of watertight input requirement in implicit representations, enabling them to work with common facial surfaces for the first time. Extensive experiments are performed to demonstrate the superiority of ImFace.