Dynamic Facial Asset and Rig Generation from a Single Scan

TL;DR

This paper introduces an automated framework that generates high-quality, personalized facial assets and rigs from a single scan, significantly reducing manual effort and hardware requirements in CG character creation.

Contribution

It presents a novel self-supervised neural network approach for inferring personalized blendshapes and dynamic textures from a single neutral scan, compatible with industry pipelines.

Findings

Robust inference across diverse subjects

High-fidelity dynamic textures and blendshapes generated

Seamless integration with existing animation pipelines

Abstract

The creation of high-fidelity computer-generated (CG) characters used in film and gaming requires intensive manual labor and a comprehensive set of facial assets to be captured with complex hardware, resulting in high cost and long production cycles. In order to simplify and accelerate this digitization process, we propose a framework for the automatic generation of high-quality dynamic facial assets, including rigs which can be readily deployed for artists to polish. Our framework takes a single scan as input to generate a set of personalized blendshapes, dynamic and physically-based textures, as well as secondary facial components (e.g., teeth and eyeballs). Built upon a facial database consisting of pore-level details, with over $4,000$ scans of varying expressions and identities, we adopt a self-supervised neural network to learn personalized blendshapes from a set of template expressions. We also model the joint distribution between identities and expressions, enabling the inference of the full set of personalized blendshapes with dynamic appearances from a single neutral input scan. Our generated personalized face rig assets are seamlessly compatible with cutting-edge industry pipelines for facial animation and rendering. We demonstrate that our framework is robust and effective by inferring on a wide range of novel subjects, and illustrate compelling rendering results while animating faces with generated customized physically-based dynamic textures.