ColorFLUX: A Structure-Color Decoupling Framework for Old Photo Colorization

TL;DR

This paper introduces ColorFLUX, a diffusion-based framework for old photo colorization that decouples structure and color, uses progressive preference learning, and incorporates semantic prompts to improve accuracy and vividness.

Contribution

The paper presents a novel structure-color decoupling diffusion model with progressive preference optimization and semantic prompts for enhanced old photo colorization.

Findings

Outperforms existing state-of-the-art colorization methods.

Produces high-quality, vivid colorizations on synthetic and real datasets.

Effectively maintains structural consistency while restoring accurate colors.

Abstract

Old photos preserve invaluable historical memories, making their restoration and colorization highly desirable. While existing restoration models can address some degradation issues like denoising and scratch removal, they often struggle with accurate colorization. This limitation arises from the unique degradation inherent in old photos, such as faded brightness and altered color hues, which are different from modern photo distributions, creating a substantial domain gap during colorization. In this paper, we propose a novel old photo colorization framework based on the generative diffusion model FLUX. Our approach introduces a structure-color decoupling strategy that separates structure preservation from color restoration, enabling accurate colorization of old photos while maintaining structural consistency. We further enhance the model with a progressive Direct Preference Optimization (Pro-DPO) strategy, which allows the model to learn subtle color preferences through coarse-to-fine transitions in color augmentation. Additionally, we address the limitations of text-based prompts by introducing visual semantic prompts, which extract fine-grained semantic information directly from old photos, helping to eliminate the color bias inherent in old photos. Experimental results on both synthetic and real datasets demonstrate that our approach outperforms existing state-of-the-art colorization methods, including closed-source commercial models, producing high-quality and vivid colorization.