Deep Colorization

TL;DR

This paper presents a fully automatic deep learning-based colorization method that leverages large-scale reference databases and advanced post-processing to produce high-quality, artifact-free colorized images efficiently.

Contribution

It introduces a novel deep learning framework for automatic colorization that integrates large-scale reference data, adaptive clustering, and bilateral filtering for superior results.

Findings

Outperforms state-of-the-art algorithms in quality and speed

Effectively handles large reference databases despite patch matching noise

Produces artifact-free, high-quality colorized images

Abstract

This paper investigates into the colorization problem which converts a grayscale image to a colorful version. This is a very difficult problem and normally requires manual adjustment to achieve artifact-free quality. For instance, it normally requires human-labelled color scribbles on the grayscale target image or a careful selection of colorful reference images (e.g., capturing the same scene in the grayscale target image). Unlike the previous methods, this paper aims at a high-quality fully-automatic colorization method. With the assumption of a perfect patch matching technique, the use of an extremely large-scale reference database (that contains sufficient color images) is the most reliable solution to the colorization problem. However, patch matching noise will increase with respect to the size of the reference database in practice. Inspired by the recent success in deep learning techniques which provide amazing modeling of large-scale data, this paper re-formulates the colorization problem so that deep learning techniques can be directly employed. To ensure artifact-free quality, a joint bilateral filtering based post-processing step is proposed. We further develop an adaptive image clustering technique to incorporate the global image information. Numerous experiments demonstrate that our method outperforms the state-of-art algorithms both in terms of quality and speed.