Facial Expression Translation using Landmark Guided GANs

TL;DR

This paper introduces LandmarkGAN, a novel GAN-based method that uses facial landmarks to translate facial expressions from a single image, outperforming existing keypoint-guided approaches.

Contribution

LandmarkGAN is the first to explicitly incorporate landmark information for expression translation using only a single image, with a two-stage end-to-end training process.

Findings

Outperforms state-of-the-art methods on four datasets.

Requires only a single image for expression translation.

Effective landmark-guided translation in diverse poses and backgrounds.

Abstract

We propose a simple yet powerful Landmark guided Generative Adversarial Network (LandmarkGAN) for the facial expression-to-expression translation using a single image, which is an important and challenging task in computer vision since the expression-to-expression translation is a non-linear and non-aligned problem. Moreover, it requires a high-level semantic understanding between the input and output images since the objects in images can have arbitrary poses, sizes, locations, backgrounds, and self-occlusions. To tackle this problem, we propose utilizing facial landmark information explicitly. Since it is a challenging problem, we split it into two sub-tasks, (i) category-guided landmark generation, and (ii) landmark-guided expression-to-expression translation. Two sub-tasks are trained in an end-to-end fashion that aims to enjoy the mutually improved benefits from the generated landmarks and expressions. Compared with current keypoint-guided approaches, the proposed LandmarkGAN only needs a single facial image to generate various expressions. Extensive experimental results on four public datasets demonstrate that the proposed LandmarkGAN achieves better results compared with state-of-the-art approaches only using a single image. The code is available at https://github.com/Ha0Tang/LandmarkGAN.