Convolutional Sketch Inversion

TL;DR

This paper presents a deep learning approach for converting face sketches into photorealistic images, utilizing a large semi-simulated dataset and achieving state-of-the-art results applicable to both computer-generated and hand-drawn sketches.

Contribution

The authors introduce a new semi-simulated dataset and a deep neural network model that outperforms existing methods in face sketch inversion, enabling photorealistic synthesis from sketches.

Findings

Achieved state-of-the-art results on sketch inversion tasks.

Effective inverting both computer-generated and hand-drawn sketches.

Potential applications in arts and forensic analysis.

Abstract

In this paper, we use deep neural networks for inverting face sketches to synthesize photorealistic face images. We first construct a semi-simulated dataset containing a very large number of computer-generated face sketches with different styles and corresponding face images by expanding existing unconstrained face data sets. We then train models achieving state-of-the-art results on both computer-generated sketches and hand-drawn sketches by leveraging recent advances in deep learning such as batch normalization, deep residual learning, perceptual losses and stochastic optimization in combination with our new dataset. We finally demonstrate potential applications of our models in fine arts and forensic arts. In contrast to existing patch-based approaches, our deep-neural-network-based approach can be used for synthesizing photorealistic face images by inverting face sketches in the wild.