ExprGAN: Facial Expression Editing with Controllable Expression Intensity

TL;DR

ExprGAN is a novel generative model that enables photo-realistic facial expression editing with controllable intensity, overcoming limitations of previous methods that require paired data or only handle categorical expressions.

Contribution

We introduce ExprGAN with an expression controller module for continuous intensity adjustment and an incremental learning scheme to handle small datasets.

Findings

Effective expression editing demonstrated on Oulu-CASIA dataset

Continuous control of expression intensity from low to high

Applications include expression transfer, image retrieval, and data augmentation

Abstract

Facial expression editing is a challenging task as it needs a high-level semantic understanding of the input face image. In conventional methods, either paired training data is required or the synthetic face resolution is low. Moreover, only the categories of facial expression can be changed. To address these limitations, we propose an Expression Generative Adversarial Network (ExprGAN) for photo-realistic facial expression editing with controllable expression intensity. An expression controller module is specially designed to learn an expressive and compact expression code in addition to the encoder-decoder network. This novel architecture enables the expression intensity to be continuously adjusted from low to high. We further show that our ExprGAN can be applied for other tasks, such as expression transfer, image retrieval, and data augmentation for training improved face expression recognition models. To tackle the small size of the training database, an effective incremental learning scheme is proposed. Quantitative and qualitative evaluations on the widely used Oulu-CASIA dataset demonstrate the effectiveness of ExprGAN.