Joint Intensity-Gradient Guided Generative Modeling for Colorization

TL;DR

This paper introduces an iterative generative model for automatic image colorization that leverages joint intensity-gradient information to improve edge preservation and reduce color overflow, outperforming existing methods.

Contribution

It proposes a novel joint intensity-gradient domain approach and a data-fidelity constraint for enhanced edge-preserving colorization in an unsupervised learning framework.

Findings

Outperforms state-of-the-art colorization methods in quantitative metrics

Achieves better edge preservation and color accuracy

Validated through extensive experiments and user studies

Abstract

This paper proposes an iterative generative model for solving the automatic colorization problem. Although previous researches have shown the capability to generate plausible color, the edge color overflow and the requirement of the reference images still exist. The starting point of the unsupervised learning in this study is the observation that the gradient map possesses latent information of the image. Therefore, the inference process of the generative modeling is conducted in joint intensity-gradient domain. Specifically, a set of intensity-gradient formed high-dimensional tensors, as the network input, are used to train a powerful noise conditional score network at the training phase. Furthermore, the joint intensity-gradient constraint in data-fidelity term is proposed to limit the degree of freedom within generative model at the iterative colorization stage, and it is conducive to edge-preserving. Extensive experiments demonstrated that the system outperformed state-of-the-art methods whether in quantitative comparisons or user study.