DNPM: A Neural Parametric Model for the Synthesis of Facial Geometric Details

TL;DR

This paper introduces DNPM, a neural parametric model that enhances 3D face modeling by synthesizing fine geometric details, enabling improved facial animation and reconstruction from limited inputs.

Contribution

The paper presents DNPM, a novel neural model for detailed facial geometry synthesis, and introduces Detailed3DMM, augmenting traditional 3DMMs with fine detail generation capabilities.

Findings

DNPM effectively captures facial details from displacement maps.

Detailed3DMM improves facial detail synthesis from identity and expression inputs.

Applications include speech-driven facial animation and face reconstruction from degraded images.

Abstract

Parametric 3D models have enabled a wide variety of computer vision and graphics tasks, such as modeling human faces, bodies and hands. In 3D face modeling, 3DMM is the most widely used parametric model, but can't generate fine geometric details solely from identity and expression inputs. To tackle this limitation, we propose a neural parametric model named DNPM for the facial geometric details, which utilizes deep neural network to extract latent codes from facial displacement maps encoding details and wrinkles. Built upon DNPM, a novel 3DMM named Detailed3DMM is proposed, which augments traditional 3DMMs by including the synthesis of facial details only from the identity and expression inputs. Moreover, we show that DNPM and Detailed3DMM can facilitate two downstream applications: speech-driven detailed 3D facial animation and 3D face reconstruction from a degraded image. Extensive experiments have shown the usefulness of DNPM and Detailed3DMM, and the progressiveness of two proposed applications.