Anatomically Constrained Implicit Face Models

TL;DR

This paper introduces an implicit neural representation for anatomically constrained face models that are faster and require less data, enabling improved shape fitting, editing, and retargeting.

Contribution

It presents a novel ensemble of implicit neural networks that model facial anatomy and surface, replacing traditional slow, data-intensive anatomical face models.

Findings

High-fidelity modeling of facial anatomy and surface.

Effective in shape fitting, editing, and retargeting tasks.

Can be used as a drop-in replacement for conventional models.

Abstract

Coordinate based implicit neural representations have gained rapid popularity in recent years as they have been successfully used in image, geometry and scene modeling tasks. In this work, we present a novel use case for such implicit representations in the context of learning anatomically constrained face models. Actor specific anatomically constrained face models are the state of the art in both facial performance capture and performance retargeting. Despite their practical success, these anatomical models are slow to evaluate and often require extensive data capture to be built. We propose the anatomical implicit face model; an ensemble of implicit neural networks that jointly learn to model the facial anatomy and the skin surface with high-fidelity, and can readily be used as a drop in replacement to conventional blendshape models. Given an arbitrary set of skin surface meshes of an actor and only a neutral shape with estimated skull and jaw bones, our method can recover a dense anatomical substructure which constrains every point on the facial surface. We demonstrate the usefulness of our approach in several tasks ranging from shape fitting, shape editing, and performance retargeting.