Learning Representations for Automatic Colorization

TL;DR

This paper presents a deep learning-based automatic image colorization system that predicts per-pixel color histograms, outperforming existing methods and enabling self-supervised visual representation learning.

Contribution

The authors introduce a novel deep network approach that predicts per-pixel color histograms for automatic colorization and explore its use for self-supervised learning.

Findings

Outperforms existing colorization methods on automatic tasks

Predicts per-pixel color histograms for flexible image generation

Enables self-supervised visual representation learning

Abstract

We develop a fully automatic image colorization system. Our approach leverages recent advances in deep networks, exploiting both low-level and semantic representations. As many scene elements naturally appear according to multimodal color distributions, we train our model to predict per-pixel color histograms. This intermediate output can be used to automatically generate a color image, or further manipulated prior to image formation. On both fully and partially automatic colorization tasks, we outperform existing methods. We also explore colorization as a vehicle for self-supervised visual representation learning.