SliderGAN: Synthesizing Expressive Face Images by Sliding 3D Blendshape Parameters

TL;DR

SliderGAN introduces a novel method for facial image editing by controlling continuous blendshape parameters, enabling flexible expression and speech modifications beyond traditional discrete models.

Contribution

The paper presents SliderGAN, a new GAN-based framework that allows continuous control of facial expressions through blendshape parameters, enhancing face editing capabilities.

Findings

Enables smooth expression transitions using sliders

Allows editing based on speech and expression blendshapes

Outperforms discrete expression models in flexibility

Abstract

Image-to-image (i2i) translation is the dense regression problem of learning how to transform an input image into an output using aligned image pairs. Remarkable progress has been made in i2i translation with the advent of Deep Convolutional Neural Networks (DCNNs) and particular using the learning paradigm of Generative Adversarial Networks (GANs). In the absence of paired images, i2i translation is tackled with one or multiple domain transformations (i.e., CycleGAN, StarGAN etc.). In this paper, we study a new problem, that of image-to-image translation, under a set of continuous parameters that correspond to a model describing a physical process. In particular, we propose the SliderGAN which transforms an input face image into a new one according to the continuous values of a statistical blendshape model of facial motion. We show that it is possible to edit a facial image according to expression and speech blendshapes, using sliders that control the continuous values of the blendshape model. This provides much more flexibility in various tasks, including but not limited to face editing, expression transfer and face neutralisation, comparing to models based on discrete expressions or action units.