ColorPeel: Color Prompt Learning with Diffusion Models via Color and Shape Disentanglement

TL;DR

ColorPeel introduces a method for learning precise color prompts in diffusion-based text-to-image models by disentangling color and shape, enabling improved color accuracy and generalization to abstract attributes.

Contribution

It proposes a novel approach to disentangle color and shape in prompt learning, enhancing the precision and versatility of T2I models for color and attribute control.

Findings

Achieves precise color generation in T2I models.

Successfully generalizes to abstract attributes like textures and materials.

Demonstrates improved control over color prompts.

Abstract

Text-to-Image (T2I) generation has made significant advancements with the advent of diffusion models. These models exhibit remarkable abilities to produce images based on textual prompts. Current T2I models allow users to specify object colors using linguistic color names. However, these labels encompass broad color ranges, making it difficult to achieve precise color matching. To tackle this challenging task, named color prompt learning, we propose to learn specific color prompts tailored to user-selected colors. Existing T2I personalization methods tend to result in color-shape entanglement. To overcome this, we generate several basic geometric objects in the target color, allowing for color and shape disentanglement during the color prompt learning. Our method, denoted as ColorPeel, successfully assists the T2I models to peel off the novel color prompts from these colored shapes. In the experiments, we demonstrate the efficacy of ColorPeel in achieving precise color generation with T2I models. Furthermore, we generalize ColorPeel to effectively learn abstract attribute concepts, including textures, materials, etc. Our findings represent a significant step towards improving precision and versatility of T2I models, offering new opportunities for creative applications and design tasks. Our project is available at https://moatifbutt.github.io/colorpeel/.