Bridging Unpaired Facial Photos And Sketches By Line-drawings

TL;DR

This paper introduces sRender, a method that uses line-drawings as an intermediary to synthesize face sketches from unpaired photos and sketches, employing neural style transfer and a stroke loss.

Contribution

The paper presents a novel unpaired face sketch synthesis approach using line-drawings as a bridge, with a new stroke loss and superior multi-style sketch generation.

Findings

Outperforms existing unpaired translation methods

Generates multi-style sketches effectively

Aligns with human artistic rendering processes

Abstract

In this paper, we propose a novel method to learn face sketch synthesis models by using unpaired data. Our main idea is bridging the photo domain $\mathcal{X}$ and the sketch domain $Y$ by using the line-drawing domain $\mathcal{Z}$. Specially, we map both photos and sketches to line-drawings by using a neural style transfer method, i.e. $F: \mathcal{X}/\mathcal{Y} \mapsto \mathcal{Z}$. Consequently, we obtain \textit{pseudo paired data} $(\mathcal{Z}, \mathcal{Y})$, and can learn the mapping $G:\mathcal{Z} \mapsto \mathcal{Y}$ in a supervised learning manner. In the inference stage, given a facial photo, we can first transfer it to a line-drawing and then to a sketch by $G \circ F$. Additionally, we propose a novel stroke loss for generating different types of strokes. Our method, termed sRender, accords well with human artists' rendering process. Experimental results demonstrate that sRender can generate multi-style sketches, and significantly outperforms existing unpaired image-to-image translation methods.