Learning Generalizable Perceptual Representations for Data-Efficient No-Reference Image Quality Assessment

TL;DR

This paper introduces a novel approach for no-reference image quality assessment that combines unsupervised low-level feature learning with high-level vision-language model fine-tuning, enabling data-efficient and zero-shot quality prediction across diverse distortions.

Contribution

It proposes a new contrastive loss for distortion-agnostic feature learning and leverages vision-language models for high-level quality assessment, enhancing generalizability and reducing annotation needs.

Findings

Outperforms existing methods on diverse datasets.

Effective in data-efficient and zero-shot scenarios.

Combines low-level and high-level features for improved accuracy.

Abstract

No-reference (NR) image quality assessment (IQA) is an important tool in enhancing the user experience in diverse visual applications. A major drawback of state-of-the-art NR-IQA techniques is their reliance on a large number of human annotations to train models for a target IQA application. To mitigate this requirement, there is a need for unsupervised learning of generalizable quality representations that capture diverse distortions. We enable the learning of low-level quality features agnostic to distortion types by introducing a novel quality-aware contrastive loss. Further, we leverage the generalizability of vision-language models by fine-tuning one such model to extract high-level image quality information through relevant text prompts. The two sets of features are combined to effectively predict quality by training a simple regressor with very few samples on a target dataset. Additionally, we design zero-shot quality predictions from both pathways in a completely blind setting. Our experiments on diverse datasets encompassing various distortions show the generalizability of the features and their superior performance in the data-efficient and zero-shot settings. Code will be made available at https://github.com/suhas-srinath/GRepQ.