Weakly-supervised Caricature Face Parsing through Domain Adaptation

TL;DR

This paper introduces a domain adaptation approach for caricature face parsing, leveraging style transfer and shape adjustment to use labeled real photo data for caricature parsing, reducing the need for extensive pixel-level annotations.

Contribution

It proposes a novel domain adaptation framework combining shape and texture transfer to enable caricature face parsing using labeled real photo datasets.

Findings

Effective caricature face parsing demonstrated on synthetic and real caricatures.

The method reduces the need for labor-intensive pixel-level annotations.

Improved accuracy over baseline methods.

Abstract

A caricature is an artistic form of a person's picture in which certain striking characteristics are abstracted or exaggerated in order to create a humor or sarcasm effect. For numerous caricature related applications such as attribute recognition and caricature editing, face parsing is an essential pre-processing step that provides a complete facial structure understanding. However, current state-of-the-art face parsing methods require large amounts of labeled data on the pixel-level and such process for caricature is tedious and labor-intensive. For real photos, there are numerous labeled datasets for face parsing. Thus, we formulate caricature face parsing as a domain adaptation problem, where real photos play the role of the source domain, adapting to the target caricatures. Specifically, we first leverage a spatial transformer based network to enable shape domain shifts. A feed-forward style transfer network is then utilized to capture texture-level domain gaps. With these two steps, we synthesize face caricatures from real photos, and thus we can use parsing ground truths of the original photos to learn the parsing model. Experimental results on the synthetic and real caricatures demonstrate the effectiveness of the proposed domain adaptation algorithm. Code is available at: https://github.com/ZJULearning/CariFaceParsing .