Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models

TL;DR

This paper introduces GRMP-IQA, a novel framework that adapts vision-language models for image quality assessment with limited data, using meta-learning and gradient regularization to achieve high accuracy.

Contribution

The paper proposes a new meta-prompt based IQA framework that efficiently adapts CLIP to IQA tasks with limited data, outperforming existing methods.

Findings

GRMP-IQA achieves competitive performance with only 20% of training data.

The framework outperforms state-of-the-art BIQA methods under limited data conditions.

Meta-learning and gradient regularization enhance model adaptation and prevent overfitting.

Abstract

Image Quality Assessment (IQA) remains an unresolved challenge in computer vision due to complex distortions, diverse image content, and limited data availability. Existing Blind IQA (BIQA) methods largely rely on extensive human annotations, which are labor-intensive and costly due to the demanding nature of creating IQA datasets. To reduce this dependency, we propose the Gradient-Regulated Meta-Prompt IQA Framework (GRMP-IQA), designed to efficiently adapt the visual-language pre-trained model, CLIP, to IQA tasks, achieving high accuracy even with limited data. GRMP-IQA consists of two core modules: (i) Meta-Prompt Pre-training Module and (ii) Quality-Aware Gradient Regularization. The Meta Prompt Pre-training Module leverages a meta-learning paradigm to pre-train soft prompts with shared meta-knowledge across different distortions, enabling rapid adaptation to various IQA tasks. On the other hand, the Quality-Aware Gradient Regularization is designed to adjust the update gradients during fine-tuning, focusing the model's attention on quality-relevant features and preventing overfitting to semantic information. Extensive experiments on standard BIQA datasets demonstrate the superior performance to the state-of-the-art BIQA methods under limited data setting. Notably, utilizing just 20% of the training data, GRMP-IQA is competitive with most existing fully supervised BIQA approaches.