Recolour What Matters: Region-Aware Colour Editing via Token-Level Diffusion

TL;DR

ColourCrafter introduces a region-aware diffusion framework for precise, controllable, and fine-grained colour editing in images, overcoming limitations of previous methods by using token-level fusion and a perceptual Lab-space loss.

Contribution

The paper presents ColourCrafter, a novel diffusion-based approach that enables structured, region-aware colour editing with improved accuracy and fidelity, supported by a new large-scale dataset.

Findings

Achieves state-of-the-art colour accuracy and controllability

Enhances perceptual fidelity in fine-grained colour edits

Demonstrates superior performance over existing methods

Abstract

Colour is one of the most perceptually salient yet least controllable attributes in image generation. Although recent diffusion models can modify object colours from user instructions, their results often deviate from the intended hue, especially for fine-grained and local edits. Early text-driven methods rely on discrete language descriptions that cannot accurately represent continuous chromatic variations. To overcome this limitation, we propose ColourCrafter, a unified diffusion framework that transforms colour editing from global tone transfer into a structured, region-aware generation process. Unlike traditional colour driven methods, ColourCrafter performs token-level fusion of RGB colour tokens and image tokens in latent space, selectively propagating colour information to semantically relevant regions while preserving structural fidelity. A perceptual Lab-space Loss further enhances pixel-level precision by decoupling luminance and chrominance and constraining edits within masked areas. Additionally, we build ColourfulSet, a largescale dataset of high-quality image pairs with continuous and diverse colour variations. Extensive experiments demonstrate that ColourCrafter achieves state-of-the-art colour accuracy, controllability and perceptual fidelity in fine-grained colour editing. Our project is available at https://yangyuqi317.github.io/ColourCrafter.github.io/.