Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network

TL;DR

This paper introduces SDGAN, a novel face sketch synthesis method that leverages semantic information and an adaptive loss to produce more accurate and realistic sketches, outperforming existing approaches.

Contribution

The paper proposes a semantic-driven GAN with global structure injection and local class re-weighting, improving face sketch synthesis quality.

Findings

Achieves state-of-the-art performance on CUFS and CUFSF datasets.

Effectively incorporates facial saliency and parsing layouts for better structure.

Enhances detail realism with a novel adaptive re-weighting loss.

Abstract

Face sketch synthesis has made significant progress with the development of deep neural networks in these years. The delicate depiction of sketch portraits facilitates a wide range of applications like digital entertainment and law enforcement. However, accurate and realistic face sketch generation is still a challenging task due to the illumination variations and complex backgrounds in the real scenes. To tackle these challenges, we propose a novel Semantic-Driven Generative Adversarial Network (SDGAN) which embeds global structure-level style injection and local class-level knowledge re-weighting. Specifically, we conduct facial saliency detection on the input face photos to provide overall facial texture structure, which could be used as a global type of prior information. In addition, we exploit face parsing layouts as the semantic-level spatial prior to enforce globally structural style injection in the generator of SDGAN. Furthermore, to enhance the realistic effect of the details, we propose a novel Adaptive Re-weighting Loss (ARLoss) which dedicates to balance the contributions of different semantic classes. Experimentally, our extensive experiments on CUFS and CUFSF datasets show that our proposed algorithm achieves state-of-the-art performance.