Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation

TL;DR

This paper introduces SEMM, a novel structure-aware morphable model that effectively captures and manipulates facial details like wrinkles, enabling realistic and semantic editing of 3D facial features.

Contribution

SEMM is the first model to incorporate wrinkle line structures and semantic editing modules into 3D face morphable models for detailed facial manipulation.

Findings

Outperforms previous methods in expression animation

Enables effective age and wrinkle line editing

Compactly represents facial details

Abstract

Morphable models are essential for the statistical modeling of 3D faces. Previous works on morphable models mostly focus on large-scale facial geometry but ignore facial details. This paper augments morphable models in representing facial details by learning a Structure-aware Editable Morphable Model (SEMM). SEMM introduces a detail structure representation based on the distance field of wrinkle lines, jointly modeled with detail displacements to establish better correspondences and enable intuitive manipulation of wrinkle structure. Besides, SEMM introduces two transformation modules to translate expression blendshape weights and age values into changes in latent space, allowing effective semantic detail editing while maintaining identity. Extensive experiments demonstrate that the proposed model compactly represents facial details, outperforms previous methods in expression animation qualitatively and quantitatively, and achieves effective age editing and wrinkle line editing of facial details. Code and model are available at https://github.com/gerwang/facial-detail-manipulation.