Mesh-Tension Driven Expression-Based Wrinkles for Synthetic Faces

TL;DR

This paper enhances synthetic face realism by introducing dynamic, expression-driven wrinkles using a mesh-tension approach, significantly improving downstream computer vision task performance.

Contribution

It presents a novel mesh-tension method to generate realistic facial wrinkles across diverse digital humans, overcoming previous limitations in detail fidelity and scalability.

Findings

Improved performance in face-related tasks with more realistic synthetic data.

Effective generation of wrinkles for unseen expressions.

Introduction of new datasets for evaluating expression deformations.

Abstract

Recent advances in synthesizing realistic faces have shown that synthetic training data can replace real data for various face-related computer vision tasks. A question arises: how important is realism? Is the pursuit of photorealism excessive? In this work, we show otherwise. We boost the realism of our synthetic faces by introducing dynamic skin wrinkles in response to facial expressions and observe significant performance improvements in downstream computer vision tasks. Previous approaches for producing such wrinkles either required prohibitive artist effort to scale across identities and expressions or were not capable of reconstructing high-frequency skin details with sufficient fidelity. Our key contribution is an approach that produces realistic wrinkles across a large and diverse population of digital humans. Concretely, we formalize the concept of mesh-tension and use it to aggregate possible wrinkles from high-quality expression scans into albedo and displacement texture maps. At synthesis, we use these maps to produce wrinkles even for expressions not represented in the source scans. Additionally, to provide a more nuanced indicator of model performance under deformations resulting from compressed expressions, we introduce the 300W-winks evaluation subset and the Pexels dataset of closed eyes and winks.