Exploring Palette based Color Guidance in Diffusion Models

TL;DR

This paper introduces a novel palette guidance method for diffusion models, significantly improving control over image color schemes in text-to-image generation by integrating color palettes as a guidance mechanism.

Contribution

It proposes a new approach that combines color palettes with prompt instructions to enhance color control in diffusion-based image generation.

Findings

Palette guidance improves color scheme accuracy

Enhanced control over background and less prominent objects

Quantitative and qualitative analysis confirms effectiveness

Abstract

With the advent of diffusion models, Text-to-Image (T2I) generation has seen substantial advancements. Current T2I models allow users to specify object colors using linguistic color names, and some methods aim to personalize color-object association through prompt learning. However, existing models struggle to provide comprehensive control over the color schemes of an entire image, especially for background elements and less prominent objects not explicitly mentioned in prompts. This paper proposes a novel approach to enhance color scheme control by integrating color palettes as a separate guidance mechanism alongside prompt instructions. We investigate the effectiveness of palette guidance by exploring various palette representation methods within a diffusion-based image colorization framework. To facilitate this exploration, we construct specialized palette-text-image datasets and conduct extensive quantitative and qualitative analyses. Our results demonstrate that incorporating palette guidance significantly improves the model's ability to generate images with desired color schemes, enabling a more controlled and refined colorization process.