Unconstrained Face Sketch Synthesis via Perception-Adaptive Network and A New Benchmark

TL;DR

This paper introduces a novel perception-adaptive network for high-quality face sketch synthesis in unconstrained conditions and presents a new challenging benchmark dataset for the task.

Contribution

The paper proposes a new end-to-end perception-adaptive network and introduces the WildSketch benchmark for unconstrained face sketch synthesis.

Findings

Achieves state-of-the-art performance on constrained and unconstrained datasets.

Demonstrates robustness to pose, expression, and illumination variations.

Provides a new benchmark dataset for future research.

Abstract

Face sketch generation has attracted much attention in the field of visual computing. However, existing methods either are limited to constrained conditions or heavily rely on various preprocessing steps to deal with in-the-wild cases. In this paper, we argue that accurately perceiving facial region and facial components is crucial for unconstrained sketch synthesis. To this end, we propose a novel Perception-Adaptive Network (PANet), which can generate high-quality face sketches under unconstrained conditions in an end-to-end scheme. Specifically, our PANet is composed of i) a Fully Convolutional Encoder for hierarchical feature extraction, ii) a Face-Adaptive Perceiving Decoder for extracting potential facial region and handling face variations, and iii) a Component-Adaptive Perceiving Module for facial component aware feature representation learning. To facilitate further researches of unconstrained face sketch synthesis, we introduce a new benchmark termed WildSketch, which contains 800 pairs of face photo-sketch with large variations in pose, expression, ethnic origin, background, and illumination. Extensive experiments demonstrate that the proposed method is capable of achieving state-of-the-art performance under both constrained and unconstrained conditions. Our source codes and the WildSketch benchmark are resealed on the project page http://lingboliu.com/unconstrained_face_sketch.html.