Exemplar-Based Image Colorization with A Learning Framework

TL;DR

This paper introduces an automatic image colorization method that combines exemplar-based and learning-based techniques, enabling diverse color styles and improved accuracy through a learned matching process and spatial post-processing.

Contribution

It proposes a hybrid colorization framework that decouples colorization and learning, using a large training set and global features to adapt to different image compositions.

Findings

Achieves comparable performance to state-of-the-art methods.

Effectively generates diverse color styles for the same gray image.

Utilizes a spatial consistency post-processing to improve color smoothness.

Abstract

Image learning and colorization are hot spots in multimedia domain. Inspired by the learning capability of humans, in this paper, we propose an automatic colorization method with a learning framework. This method can be viewed as a hybrid of exemplar-based and learning-based method, and it decouples the colorization process and learning process so as to generate various color styles for the same gray image. The matching process in the exemplar-based colorization method can be regarded as a parameterized function, and we employ a large amount of color images as the training samples to fit the parameters. During the training process, the color images are the ground truths, and we learn the optimal parameters for the matching process by minimizing the errors in terms of the parameters for the matching function. To deal with images with various compositions, a global feature is introduced, which can be used to classify the images with respect to their compositions, and then learn the optimal matching parameters for each image category individually. What's more, a spatial consistency based post-processing is design to smooth the extracted color information from the reference image to remove matching errors. Extensive experiments are conducted to verify the effectiveness of the method, and it achieves comparable performance against the state-of-the-art colorization algorithms.