Learning Multi-dimensional Human Preference for Text-to-Image Generation

TL;DR

This paper introduces a multi-dimensional preference scoring model (MPS) that captures diverse human preferences in text-to-image generation, outperforming existing metrics by considering aesthetics, semantic alignment, detail quality, and overall assessment.

Contribution

The paper presents the first multi-dimensional preference score for evaluating text-to-image models, utilizing a large human preference dataset and a preference condition module on CLIP.

Findings

MPS outperforms existing metrics across multiple datasets and dimensions.

The dataset contains over 900,000 human preference choices on 607,541 images.

MPS effectively captures diverse human preferences in image evaluation.

Abstract

Current metrics for text-to-image models typically rely on statistical metrics which inadequately represent the real preference of humans. Although recent work attempts to learn these preferences via human annotated images, they reduce the rich tapestry of human preference to a single overall score. However, the preference results vary when humans evaluate images with different aspects. Therefore, to learn the multi-dimensional human preferences, we propose the Multi-dimensional Preference Score (MPS), the first multi-dimensional preference scoring model for the evaluation of text-to-image models. The MPS introduces the preference condition module upon CLIP model to learn these diverse preferences. It is trained based on our Multi-dimensional Human Preference (MHP) Dataset, which comprises 918,315 human preference choices across four dimensions (i.e., aesthetics, semantic alignment, detail quality and overall assessment) on 607,541 images. The images are generated by a wide range of latest text-to-image models. The MPS outperforms existing scoring methods across 3 datasets in 4 dimensions, enabling it a promising metric for evaluating and improving text-to-image generation.