Diffusing Colors: Image Colorization with Text Guided Diffusion
Nir Zabari, Aharon Azulay, Alexey Gorkor, Tavi Halperin, Ohad Fried

TL;DR
This paper introduces a novel image colorization method using diffusion models guided by text prompts, enhancing control, semantic accuracy, and visual quality in grayscale image colorization.
Contribution
The paper proposes a new diffusion-based colorization framework with text guidance and a CLIP-based ranking system, improving controllability and output quality over existing methods.
Findings
Outperforms existing colorization techniques in visual quality and semantic coherence.
Enables fine-tuned control over color vividness through text prompts.
Demonstrates effectiveness in color enhancement and historical image colorization.
Abstract
The colorization of grayscale images is a complex and subjective task with significant challenges. Despite recent progress in employing large-scale datasets with deep neural networks, difficulties with controllability and visual quality persist. To tackle these issues, we present a novel image colorization framework that utilizes image diffusion techniques with granular text prompts. This integration not only produces colorization outputs that are semantically appropriate but also greatly improves the level of control users have over the colorization process. Our method provides a balance between automation and control, outperforming existing techniques in terms of visual quality and semantic coherence. We leverage a pretrained generative Diffusion Model, and show that we can finetune it for the colorization task without losing its generative power or attention to text prompts.…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Image Enhancement Techniques · Advanced Image Processing Techniques
MethodsDiffusion · Colorization
