TIC: Text-Guided Image Colorization

TL;DR

This paper introduces TIC, a novel deep learning model that incorporates textual descriptions as auxiliary input to improve the accuracy and realism of image colorization, addressing the challenge of ill-posed colorization tasks.

Contribution

It is one of the first models to integrate textual conditioning into the image colorization process, enhancing color fidelity and realism.

Findings

Outperforms existing colorization methods both qualitatively and quantitatively.

Utilizes textual descriptions to guide the colorization process effectively.

Demonstrates improved color accuracy and scene consistency.

Abstract

Image colorization is a well-known problem in computer vision. However, due to the ill-posed nature of the task, image colorization is inherently challenging. Though several attempts have been made by researchers to make the colorization pipeline automatic, these processes often produce unrealistic results due to a lack of conditioning. In this work, we attempt to integrate textual descriptions as an auxiliary condition, along with the grayscale image that is to be colorized, to improve the fidelity of the colorization process. To the best of our knowledge, this is one of the first attempts to incorporate textual conditioning in the colorization pipeline. To do so, we have proposed a novel deep network that takes two inputs (the grayscale image and the respective encoded text description) and tries to predict the relevant color gamut. As the respective textual descriptions contain color information of the objects present in the scene, the text encoding helps to improve the overall quality of the predicted colors. We have evaluated our proposed model using different metrics and found that it outperforms the state-of-the-art colorization algorithms both qualitatively and quantitatively.