Face Destylization

TL;DR

This paper introduces a neural network that effectively reverses artistic style transfer on portraits, restoring realistic faces from stylized images, which benefits identity analysis and perception tasks.

Contribution

We propose a novel Face Destylization Neural Network (FDNN) that restores realistic faces from stylized portraits, including paintings, using a style removal network and adversarial training.

Findings

Successfully restores realistic faces from stylized images.

Effective on synthetic and real paintings.

Outperforms existing methods in face recovery quality.

Abstract

Numerous style transfer methods which produce artistic styles of portraits have been proposed to date. However, the inverse problem of converting the stylized portraits back into realistic faces is yet to be investigated thoroughly. Reverting an artistic portrait to its original photo-realistic face image has potential to facilitate human perception and identity analysis. In this paper, we propose a novel Face Destylization Neural Network (FDNN) to restore the latent photo-realistic faces from the stylized ones. We develop a Style Removal Network composed of convolutional, fully-connected and deconvolutional layers. The convolutional layers are designed to extract facial components from stylized face images. Consecutively, the fully-connected layer transfers the extracted feature maps of stylized images into the corresponding feature maps of real faces and the deconvolutional layers generate real faces from the transferred feature maps. To enforce the destylized faces to be similar to authentic face images, we employ a discriminative network, which consists of convolutional and fully connected layers. We demonstrate the effectiveness of our network by conducting experiments on an extensive set of synthetic images. Furthermore, we illustrate our network can recover faces from stylized portraits and real paintings for which the stylized data was unavailable during the training phase.