BigColor: Colorization using a Generative Color Prior for Natural Images

TL;DR

BigColor introduces a novel colorization method that leverages a generative color prior focused on color synthesis, enabling vivid, diverse, and robust colorization of complex in-the-wild images with a single forward pass.

Contribution

The paper presents a new generative color prior and an encoder-generator network that expands the colorization capabilities for complex images beyond existing methods.

Findings

Outperforms existing colorization methods on in-the-wild images

Supports arbitrary resolutions and multi-modal colorization

Enables robust colorization in a single forward pass

Abstract

For realistic and vivid colorization, generative priors have recently been exploited. However, such generative priors often fail for in-the-wild complex images due to their limited representation space. In this paper, we propose BigColor, a novel colorization approach that provides vivid colorization for diverse in-the-wild images with complex structures. While previous generative priors are trained to synthesize both image structures and colors, we learn a generative color prior to focus on color synthesis given the spatial structure of an image. In this way, we reduce the burden of synthesizing image structures from the generative prior and expand its representation space to cover diverse images. To this end, we propose a BigGAN-inspired encoder-generator network that uses a spatial feature map instead of a spatially-flattened BigGAN latent code, resulting in an enlarged representation space. Our method enables robust colorization for diverse inputs in a single forward pass, supports arbitrary input resolutions, and provides multi-modal colorization results. We demonstrate that BigColor significantly outperforms existing methods especially on in-the-wild images with complex structures.