CLIP-AGIQA: Boosting the Performance of AI-Generated Image Quality Assessment with CLIP

TL;DR

This paper introduces CLIP-AGIQA, a novel CLIP-based model that significantly improves the accuracy of AI-generated image quality assessment by leveraging multi-category prompts and extensive visual-textual knowledge.

Contribution

The paper presents a new CLIP-based regression model with multi-category learnable prompts for enhanced quality assessment of generated images, outperforming existing models.

Findings

CLIP-AGIQA achieves state-of-the-art results on AGIQA-3K and AIGCIQA2023 benchmarks.

The model effectively utilizes CLIP's visual and textual knowledge for image quality evaluation.

Multi-category learnable prompts enhance the model's ability to assess diverse generated images.

Abstract

With the rapid development of generative technologies, AI-Generated Images (AIGIs) have been widely applied in various aspects of daily life. However, due to the immaturity of the technology, the quality of the generated images varies, so it is important to develop quality assessment techniques for the generated images. Although some models have been proposed to assess the quality of generated images, they are inadequate when faced with the ever-increasing and diverse categories of generated images. Consequently, the development of more advanced and effective models for evaluating the quality of generated images is urgently needed. Recent research has explored the significant potential of the visual language model CLIP in image quality assessment, finding that it performs well in evaluating the quality of natural images. However, its application to generated images has not been thoroughly investigated. In this paper, we build on this idea and further explore the potential of CLIP in evaluating the quality of generated images. We design CLIP-AGIQA, a CLIP-based regression model for quality assessment of generated images, leveraging rich visual and textual knowledge encapsulated in CLIP. Particularly, we implement multi-category learnable prompts to fully utilize the textual knowledge in CLIP for quality assessment. Extensive experiments on several generated image quality assessment benchmarks, including AGIQA-3K and AIGCIQA2023, demonstrate that CLIP-AGIQA outperforms existing IQA models, achieving excellent results in evaluating the quality of generated images.