PKU-I2IQA: An Image-to-Image Quality Assessment Database for AI Generated Images

TL;DR

This paper introduces PKU-I2IQA, a new human perception-based database for assessing image-to-image AI-generated image quality, along with benchmark models to evaluate and compare image quality assessment methods.

Contribution

The paper creates the first comprehensive I2IQA database for AI-generated images and proposes two benchmark models for quality assessment, addressing a gap in existing datasets.

Findings

The PKU-I2IQA database contains diverse human-labeled quality scores.

Benchmark models outperform existing methods on the new database.

The database and benchmarks are publicly available for future research.

Abstract

As image generation technology advances, AI-based image generation has been applied in various fields and Artificial Intelligence Generated Content (AIGC) has garnered widespread attention. However, the development of AI-based image generative models also brings new problems and challenges. A significant challenge is that AI-generated images (AIGI) may exhibit unique distortions compared to natural images, and not all generated images meet the requirements of the real world. Therefore, it is of great significance to evaluate AIGIs more comprehensively. Although previous work has established several human perception-based AIGC image quality assessment (AIGCIQA) databases for text-generated images, the AI image generation technology includes scenarios like text-to-image and image-to-image, and assessing only the images generated by text-to-image models is insufficient. To address this issue, we establish a human perception-based image-to-image AIGCIQA database, named PKU-I2IQA. We conduct a well-organized subjective experiment to collect quality labels for AIGIs and then conduct a comprehensive analysis of the PKU-I2IQA database. Furthermore, we have proposed two benchmark models: NR-AIGCIQA based on the no-reference image quality assessment method and FR-AIGCIQA based on the full-reference image quality assessment method. Finally, leveraging this database, we conduct benchmark experiments and compare the performance of the proposed benchmark models. The PKU-I2IQA database and benchmarks will be released to facilitate future research on \url{https://github.com/jiquan123/I2IQA}.