Semantic Facial Expression Editing using Autoencoded Flow

TL;DR

This paper introduces an automatic facial expression editing method that combines flow-based manipulation with VAEs, enabling high-quality, realistic expression changes and interpolation in images.

Contribution

The paper proposes a novel approach integrating flow-based face manipulation with VAEs for improved expression editing and interpolation.

Findings

Produces higher perceptual quality images than previous methods

Enables simple latent vector arithmetic for expression editing

Effective on both single-image editing and interpolation tasks

Abstract

High-level manipulation of facial expressions in images --- such as changing a smile to a neutral expression --- is challenging because facial expression changes are highly non-linear, and vary depending on the appearance of the face. We present a fully automatic approach to editing faces that combines the advantages of flow-based face manipulation with the more recent generative capabilities of Variational Autoencoders (VAEs). During training, our model learns to encode the flow from one expression to another over a low-dimensional latent space. At test time, expression editing can be done simply using latent vector arithmetic. We evaluate our methods on two applications: 1) single-image facial expression editing, and 2) facial expression interpolation between two images. We demonstrate that our method generates images of higher perceptual quality than previous VAE and flow-based methods.