Learning Spatially Decoupled Color Representations for Facial Image Colorization

TL;DR

This paper introduces FCNet, a facial image colorization framework that learns decoupled color representations for facial components, improving colorization quality especially for faces, by leveraging facial priors and augmentation strategies.

Contribution

The paper proposes a novel decoupled color representation approach for facial components guided by face parsing maps, enhancing facial colorization performance over existing methods.

Findings

Outperforms existing methods in various scenarios

Effective with no, single, or multiple reference images

Uses facial priors and augmentation for better learning

Abstract

Image colorization methods have shown prominent performance on natural images. However, since humans are more sensitive to faces, existing methods are insufficient to meet the demands when applied to facial images, typically showing unnatural and uneven colorization results. In this paper, we investigate the facial image colorization task and find that the problems with facial images can be attributed to an insufficient understanding of facial components. As a remedy, by introducing facial component priors, we present a novel facial image colorization framework dubbed FCNet. Specifically, we learn a decoupled color representation for each face component (e.g., lips, skin, eyes, and hair) under the guidance of face parsing maps. A chromatic and spatial augmentation strategy is presented to facilitate the learning procedure, which requires only grayscale and color facial image pairs. After training, the presented FCNet can be naturally applied to facial image colorization with single or multiple reference images. To expand the application paradigms to scenarios with no reference images, we further train two alternative modules, which predict the color representations from the grayscale input or a random seed, respectively. Extensive experiments show that our method can perform favorably against existing methods in various application scenarios (i.e., no-, single-, and multi-reference facial image colorization). The source code and pre-trained models will be publicly available.