Towards Vivid and Diverse Image Colorization with Generative Color Prior

TL;DR

This paper introduces GCP-Colorization, a novel image colorization method that uses a pretrained GAN's rich color priors to produce vivid, diverse, and controllable colorizations with high quality and interpretability.

Contribution

It leverages a pretrained GAN for automatic, diverse, and controllable image colorization by incorporating generative color priors through feature modulation.

Findings

Achieves superior colorization quality compared to previous methods.

Enables diverse results by modifying GAN latent codes.

Provides interpretable and smooth color transition controls.

Abstract

Colorization has attracted increasing interest in recent years. Classic reference-based methods usually rely on external color images for plausible results. A large image database or online search engine is inevitably required for retrieving such exemplars. Recent deep-learning-based methods could automatically colorize images at a low cost. However, unsatisfactory artifacts and incoherent colors are always accompanied. In this work, we propose GCP-Colorization that leverages the rich and diverse color priors encapsulated in a pretrained Generative Adversarial Networks (GAN) for automatic colorization. Specifically, we first "retrieve" matched features (similar to exemplars) via a GAN encoder and then incorporate these features into the colorization process with feature modulations. Thanks to the powerful generative color prior (GCP) and delicate designs, our GCP-Colorization could produce vivid colors with a single forward pass. Moreover, it is highly convenient to obtain diverse results by modifying GAN latent codes. GCP-Colorization also inherits the merit of interpretable controls of GANs and could attain controllable and smooth transitions by walking through GAN latent space. Extensive experiments and user studies demonstrate that GCP-Colorization achieves superior performance than previous works. Codes are available at https://github.com/ToTheBeginning/GCP-Colorization.