Too Vivid to Be Real? Benchmarking and Calibrating Generative Color Fidelity

TL;DR

This paper introduces a new dataset and metric for objectively evaluating and improving color fidelity in realistic text-to-image generation, addressing biases in existing evaluation methods that favor overly vivid images.

Contribution

It presents the Color Fidelity Dataset and Metric, a training-free refinement method, and a progressive framework for assessing and enhancing color realism in T2I generation.

Findings

CFD contains over 1.3 million images with varying color realism levels.

CFM employs a multimodal encoder to learn perceptual color fidelity.

CFR adaptively improves color authenticity without additional training.

Abstract

Recent advances in text-to-image (T2I) generation have greatly improved visual quality, yet producing images that appear visually authentic to real-world photography remains challenging. This is partly due to biases in existing evaluation paradigms: human ratings and preference-trained metrics often favor visually vivid images with exaggerated saturation and contrast, which make generations often too vivid to be real even when prompted for realistic-style images. To address this issue, we present Color Fidelity Dataset (CFD) and Color Fidelity Metric (CFM) for objective evaluation of color fidelity in realistic-style generations. CFD contains over 1.3M real and synthetic images with ordered levels of color realism, while CFM employs a multimodal encoder to learn perceptual color fidelity. In addition, we propose a training-free Color Fidelity Refinement (CFR) that adaptively modulates spatial-temporal guidance scale in generation, thereby enhancing color authenticity. Together, CFD supports CFM for assessment, whose learned attention further guides CFR to refine T2I fidelity, forming a progressive framework for assessing and improving color fidelity in realistic-style T2I generation. The dataset and code are available at https://github.com/ZhengyaoFang/CFM.