Semantics-Preserving Sketch Embedding for Face Generation

TL;DR

This paper proposes a novel sketch embedding method that preserves semantic and geometric details in face image generation from sketches, improving accuracy and generalization over previous approaches.

Contribution

It introduces a W-W+ encoder architecture with semantic supervision and an automatic sketch semantic interpretation method for enhanced face image synthesis.

Findings

Outperforms existing methods in semantics preservation

Demonstrates strong generalization to hand-drawn sketches

Achieves detailed and semantically consistent face images

Abstract

With recent advances in image-to-image translation tasks, remarkable progress has been witnessed in generating face images from sketches. However, existing methods frequently fail to generate images with details that are semantically and geometrically consistent with the input sketch, especially when various decoration strokes are drawn. To address this issue, we introduce a novel W-W+ encoder architecture to take advantage of the high expressive power of W+ space and semantic controllability of W space. We introduce an explicit intermediate representation for sketch semantic embedding. With a semantic feature matching loss for effective semantic supervision, our sketch embedding precisely conveys the semantics in the input sketches to the synthesized images. Moreover, a novel sketch semantic interpretation approach is designed to automatically extract semantics from vectorized sketches. We conduct extensive experiments on both synthesized sketches and hand-drawn sketches, and the results demonstrate the superiority of our method over existing approaches on both semantics-preserving and generalization ability.