MultiColor: Image Colorization by Learning from Multiple Color Spaces

TL;DR

MultiColor introduces a novel image colorization method that leverages multiple color spaces and their complementarity, employing dedicated modules and a complementary network to produce more accurate and visually pleasing colorized images.

Contribution

The paper proposes a new learning-based approach that combines multiple color spaces for image colorization, enhancing performance by exploiting their complementary characteristics.

Findings

Outperforms state-of-the-art methods on real-world datasets.

Utilizes multiple color spaces to improve colorization quality.

Demonstrates the effectiveness of dedicated modules and a complementary network.

Abstract

Deep networks have shown impressive performance in the image restoration tasks, such as image colorization. However, we find that previous approaches rely on the digital representation from single color model with a specific mapping function, a.k.a., color space, during the colorization pipeline. In this paper, we first investigate the modeling of different color spaces, and find each of them exhibiting distinctive characteristics with unique distribution of colors. The complementarity among multiple color spaces leads to benefits for the image colorization task. We present MultiColor, a new learning-based approach to automatically colorize grayscale images that combines clues from multiple color spaces. Specifically, we employ a set of dedicated colorization modules for individual color space. Within each module, a transformer decoder is first employed to refine color query embeddings and then a color mapper produces color channel prediction using the embeddings and semantic features. With these predicted color channels representing various color spaces, a complementary network is designed to exploit the complementarity and generate pleasing and reasonable colorized images. We conduct extensive experiments on real-world datasets, and the results demonstrate superior performance over the state-of-the-arts.