CariMe: Unpaired Caricature Generation with Multiple Exaggerations

TL;DR

This paper introduces CariMe, a novel unpaired caricature generation method that models exaggerations at the distribution level, enabling diverse and detailed caricatures with multiple exaggerations from a single photo.

Contribution

It proposes the first distribution-level deformation modeling approach for unpaired caricature generation, introducing a multi-exaggeration warper and deformation-field-based warping for detailed exaggerations.

Findings

Outperforms state-of-the-art methods in caricature quality.

Generates diverse exaggerations from a single input photo.

Shows superior results in perceptual studies.

Abstract

Caricature generation aims to translate real photos into caricatures with artistic styles and shape exaggerations while maintaining the identity of the subject. Different from the generic image-to-image translation, drawing a caricature automatically is a more challenging task due to the existence of various spacial deformations. Previous caricature generation methods are obsessed with predicting definite image warping from a given photo while ignoring the intrinsic representation and distribution for exaggerations in caricatures. This limits their ability on diverse exaggeration generation. In this paper, we generalize the caricature generation problem from instance-level warping prediction to distribution-level deformation modeling. Based on this assumption, we present the first exploration for unpaired CARIcature generation with Multiple Exaggerations (CariMe). Technically, we propose a Multi-exaggeration Warper network to learn the distribution-level mapping from photo to facial exaggerations. This makes it possible to generate diverse and reasonable exaggerations from randomly sampled warp codes given one input photo. To better represent the facial exaggeration and produce fine-grained warping, a deformation-field-based warping method is also proposed, which helps us to capture more detailed exaggerations than other point-based warping methods. Experiments and two perceptual studies prove the superiority of our method comparing with other state-of-the-art methods, showing the improvement of our work on caricature generation.