Vision Language Modeling of Content, Distortion and Appearance for Image Quality Assessment

TL;DR

This paper introduces SLIQUE, a comprehensive blind image quality assessment model that leverages vision-language and contrastive learning, trained on a large annotated database, to effectively evaluate image quality considering content, distortion, and appearance.

Contribution

The paper presents a novel joint vision-language and contrastive learning framework for IQA, along with a large annotated database, advancing the understanding of high-level image quality attributes.

Findings

SLIQUE outperforms existing IQA models in experiments.

The TADAC database enables detailed annotation of image quality factors.

The framework effectively captures semantic, distortion, and appearance information.

Abstract

The visual quality of an image is confounded by a number of intertwined factors including its semantic content, distortion characteristics and appearance properties such as brightness, contrast, sharpness, and colourfulness. Distilling high level knowledge about all these quality bearing attributes is crucial for developing objective Image Quality Assessment (IQA).While existing solutions have modeled some of these aspects, a comprehensive solution that involves all these important quality related attributes has not yet been developed. In this paper, we present a new blind IQA (BIQA) model termed Self-supervision and Vision-Language supervision Image QUality Evaluator (SLIQUE) that features a joint vision-language and visual contrastive representation learning framework for acquiring high level knowledge about the images semantic contents, distortion characteristics and appearance properties for IQA. For training SLIQUE, we have developed a systematic approach to constructing a first of its kind large image database annotated with all three categories of quality relevant texts. The Text Annotated Distortion, Appearance and Content (TADAC) database has over 1.6 million images annotated with textual descriptions of their semantic contents, distortion characteristics and appearance properties. The method for constructing TADAC and the database itself will be particularly useful for exploiting vision-language modeling for advanced IQA applications. Extensive experimental results show that SLIQUE has superior performances over state of the art, demonstrating the soundness of its design principle and the effectiveness of its implementation.