DP-IQA: Utilizing Diffusion Prior for Blind Image Quality Assessment in the Wild

TL;DR

DP-IQA leverages pre-trained diffusion models to improve blind image quality assessment in complex real-world scenarios, achieving state-of-the-art results with enhanced generalization and reduced model complexity.

Contribution

This work introduces the first application of pre-trained diffusion priors for blind IQA, utilizing Stable Diffusion features and distilling knowledge into a lightweight model for better performance.

Findings

Achieves state-of-the-art results on in-the-wild IQA datasets.

Demonstrates superior generalization compared to existing methods.

Reduces model complexity while maintaining high performance.

Abstract

Blind image quality assessment (IQA) in the wild, which assesses the quality of images with complex authentic distortions and no reference images, presents significant challenges. Given the difficulty in collecting large-scale training data, leveraging limited data to develop a model with strong generalization remains an open problem. Motivated by the robust image perception capabilities of pre-trained text-to-image (T2I) diffusion models, we propose a novel IQA method, diffusion priors-based IQA (DP-IQA), to utilize the T2I model's prior for improved performance and generalization ability. Specifically, we utilize pre-trained Stable Diffusion as the backbone, extracting multi-level features from the denoising U-Net guided by prompt embeddings through a tunable text adapter. Simultaneously, an image adapter compensates for information loss introduced by the lossy pre-trained encoder. Unlike T2I models that require full image distribution modeling, our approach targets image quality assessment, which inherently requires fewer parameters. To improve applicability, we distill the knowledge into a lightweight CNN-based student model, significantly reducing parameters while maintaining or even enhancing generalization performance. Experimental results demonstrate that DP-IQA achieves state-of-the-art performance on various in-the-wild datasets, highlighting the superior generalization capability of T2I priors in blind IQA tasks. To our knowledge, DP-IQA is the first method to apply pre-trained diffusion priors in blind IQA. Codes and checkpoints are available at https://github.com/RomGai/DP-IQA.