Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Dataset

TL;DR

This paper introduces DepictQA-Wild, a large-scale multi-modal dataset and model for descriptive image quality assessment that improves performance across diverse real-world scenarios by leveraging a comprehensive dataset and multi-task framework.

Contribution

The paper presents a new large-scale dataset DQ-495K and a multi-functional IQA model, DepictQA-Wild, addressing limitations of prior methods and enhancing descriptive image quality assessment.

Findings

DepictQA-Wild outperforms traditional and prior VLM-based IQA models.

The dataset enables better handling of resolution and quality issues.

The model is effective in real-world image assessment tasks.

Abstract

With the rapid advancement of Vision Language Models (VLMs), VLM-based Image Quality Assessment (IQA) seeks to describe image quality linguistically to align with human expression and capture the multifaceted nature of IQA tasks. However, current methods are still far from practical usage. First, prior works focus narrowly on specific sub-tasks or settings, which do not align with diverse real-world applications. Second, their performance is sub-optimal due to limitations in dataset coverage, scale, and quality. To overcome these challenges, we introduce the enhanced Depicted image Quality Assessment model (DepictQA-Wild). Our method includes a multi-functional IQA task paradigm that encompasses both assessment and comparison tasks, brief and detailed responses, full-reference and non-reference scenarios. We introduce a ground-truth-informed dataset construction approach to enhance data quality, and scale up the dataset to 495K under the brief-detail joint framework. Consequently, we construct a comprehensive, large-scale, and high-quality dataset, named DQ-495K. We also retain image resolution during training to better handle resolution-related quality issues, and estimate a confidence score that is helpful to filter out low-quality responses. Experimental results demonstrate that DepictQA-Wild significantly outperforms traditional score-based methods, prior VLM-based IQA models, and proprietary GPT-4V in distortion identification, instant rating, and reasoning tasks. Our advantages are further confirmed by real-world applications including assessing the web-downloaded images and ranking model-processed images. Codes, datasets, and model weights have been released in https://depictqa.github.io/.