Enhancing Reward Models for High-quality Image Generation: Beyond Text-Image Alignment

TL;DR

This paper introduces a new evaluation score, ICT, and an HP score model to better align image generation with human aesthetic preferences, surpassing existing text-image alignment methods in accuracy and quality.

Contribution

It proposes a novel evaluation metric and a new preference model trained on image modality, improving image aesthetic assessment beyond traditional text-image alignment.

Findings

ICT score surpasses existing evaluation metrics by over 10% in accuracy.

HP score model enhances image aesthetics and detail quality.

The approach improves state-of-the-art text-to-image model optimization.

Abstract

Contemporary image generation systems have achieved high fidelity and superior aesthetic quality beyond basic text-image alignment. However, existing evaluation frameworks have failed to evolve in parallel. This study reveals that human preference reward models fine-tuned based on CLIP and BLIP architectures have inherent flaws: they inappropriately assign low scores to images with rich details and high aesthetic value, creating a significant discrepancy with actual human aesthetic preferences. To address this issue, we design a novel evaluation score, ICT (Image-Contained-Text) score, that achieves and surpasses the objectives of text-image alignment by assessing the degree to which images represent textual content. Building upon this foundation, we further train an HP (High-Preference) score model using solely the image modality to enhance image aesthetics and detail quality while maintaining text-image alignment. Experiments demonstrate that the proposed evaluation model improves scoring accuracy by over 10\% compared to existing methods, and achieves significant results in optimizing state-of-the-art text-to-image models. This research provides theoretical and empirical support for evolving image generation technology toward higher-order human aesthetic preferences. Code is available at https://github.com/BarretBa/ICTHP.