StyleCariGAN: Caricature Generation via StyleGAN Feature Map Modulation

TL;DR

StyleCariGAN is a novel framework that uses StyleGAN feature map modulation to automatically generate detailed caricatures with controllable shape exaggeration and style, outperforming existing methods.

Contribution

It introduces shape exaggeration blocks for StyleGAN to produce realistic caricatures with adjustable shape exaggeration and style, advancing caricature generation techniques.

Findings

Produces realistic, detailed caricatures with shape exaggeration control.

Outperforms current state-of-the-art caricature methods.

Supports additional StyleGAN-based image manipulations like expression control.

Abstract

We present a caricature generation framework based on shape and style manipulation using StyleGAN. Our framework, dubbed StyleCariGAN, automatically creates a realistic and detailed caricature from an input photo with optional controls on shape exaggeration degree and color stylization type. The key component of our method is shape exaggeration blocks that are used for modulating coarse layer feature maps of StyleGAN to produce desirable caricature shape exaggerations. We first build a layer-mixed StyleGAN for photo-to-caricature style conversion by swapping fine layers of the StyleGAN for photos to the corresponding layers of the StyleGAN trained to generate caricatures. Given an input photo, the layer-mixed model produces detailed color stylization for a caricature but without shape exaggerations. We then append shape exaggeration blocks to the coarse layers of the layer-mixed model and train the blocks to create shape exaggerations while preserving the characteristic appearances of the input. Experimental results show that our StyleCariGAN generates realistic and detailed caricatures compared to the current state-of-the-art methods. We demonstrate StyleCariGAN also supports other StyleGAN-based image manipulations, such as facial expression control.