S2FGAN: Semantically Aware Interactive Sketch-to-Face Translation

TL;DR

S2FGAN is a novel sketch-to-face translation framework that enhances controllability and diversity in facial attribute editing by utilizing a dual latent space GAN approach, allowing attribute manipulation from simple sketches.

Contribution

The paper introduces a new GAN-based method that enables flexible, attribute-specific face editing from sketches without relying on reference images, improving control and diversity.

Findings

Outperforms state-of-the-art methods in attribute manipulation

Provides greater control over attribute intensity

Effectively manipulates multiple face attributes from sketches

Abstract

Interactive facial image manipulation attempts to edit single and multiple face attributes using a photo-realistic face and/or semantic mask as input. In the absence of the photo-realistic image (only sketch/mask available), previous methods only retrieve the original face but ignore the potential of aiding model controllability and diversity in the translation process. This paper proposes a sketch-to-image generation framework called S2FGAN, aiming to improve users' ability to interpret and flexibility of face attribute editing from a simple sketch. The proposed framework modifies the constrained latent space semantics trained on Generative Adversarial Networks (GANs). We employ two latent spaces to control the face appearance and adjust the desired attributes of the generated face. Instead of constraining the translation process by using a reference image, the users can command the model to retouch the generated images by involving the semantic information in the generation process. In this way, our method can manipulate single or multiple face attributes by only specifying attributes to be changed. Extensive experimental results on CelebAMask-HQ dataset empirically shows our superior performance and effectiveness on this task. Our method successfully outperforms state-of-the-art methods on attribute manipulation by exploiting greater control of attribute intensity.