Region Based Adversarial Synthesis of Facial Action Units

TL;DR

This paper presents LAC-GAN, a novel face expression synthesis method that uses local AU rules and attention mechanisms to generate realistic facial expressions from AU labels, even with unpaired training data.

Contribution

The paper introduces LAC-GAN, a new AU-level facial expression synthesis approach that effectively handles unpaired data and improves realism and control over facial expressions.

Findings

LAC-GAN produces high-quality, photo-realistic facial expressions.

The method effectively utilizes unpaired training data.

Extensive evaluations confirm the approach's superiority.

Abstract

Facial expression synthesis or editing has recently received increasing attention in the field of affective computing and facial expression modeling. However, most existing facial expression synthesis works are limited in paired training data, low resolution, identity information damaging, and so on. To address those limitations, this paper introduces a novel Action Unit (AU) level facial expression synthesis method called Local Attentive Conditional Generative Adversarial Network (LAC-GAN) based on face action units annotations. Given desired AU labels, LAC-GAN utilizes local AU regional rules to control the status of each AU and attentive mechanism to combine several of them into the whole photo-realistic facial expressions or arbitrary facial expressions. In addition, unpaired training data is utilized in our proposed method to train the manipulation module with the corresponding AU labels, which learns a mapping between a facial expression manifold. Extensive qualitative and quantitative evaluations are conducted on the commonly used BP4D dataset to verify the effectiveness of our proposed AU synthesis method.