User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks

TL;DR

This paper introduces a novel deep conditional adversarial network that improves anime line art colorization by enhancing realism, shading accuracy, and generalization to diverse line arts using a new architecture and datasets.

Contribution

It proposes a deep conditional adversarial architecture with a local features network and perceptual loss, improving realism and generalization in anime line art colorization.

Findings

Generated images are more realistic and precise.

The model generalizes well to 'in the wild' line arts.

New datasets for training and benchmarking are introduced.

Abstract

Scribble colors based line art colorization is a challenging computer vision problem since neither greyscale values nor semantic information is presented in line arts, and the lack of authentic illustration-line art training pairs also increases difficulty of model generalization. Recently, several Generative Adversarial Nets (GANs) based methods have achieved great success. They can generate colorized illustrations conditioned on given line art and color hints. However, these methods fail to capture the authentic illustration distributions and are hence perceptually unsatisfying in the sense that they often lack accurate shading. To address these challenges, we propose a novel deep conditional adversarial architecture for scribble based anime line art colorization. Specifically, we integrate the conditional framework with WGAN-GP criteria as well as the perceptual loss to enable us to robustly train a deep network that makes the synthesized images more natural and real. We also introduce a local features network that is independent of synthetic data. With GANs conditioned on features from such network, we notably increase the generalization capability over "in the wild" line arts. Furthermore, we collect two datasets that provide high-quality colorful illustrations and authentic line arts for training and benchmarking. With the proposed model trained on our illustration dataset, we demonstrate that images synthesized by the presented approach are considerably more realistic and precise than alternative approaches.