DSMix: Distortion-Induced Sensitivity Map Based Pre-training for No-Reference Image Quality Assessment

TL;DR

DSMix introduces a novel data augmentation method using distortion sensitivity maps for pre-training in no-reference image quality assessment, significantly improving model performance without fine-tuning.

Contribution

The paper proposes DSMix, a new data augmentation technique leveraging distortion sensitivity maps for pre-training in IQA, enhancing generalization and predictive accuracy.

Findings

DSMix improves IQA prediction accuracy on multiple datasets.

DSMix enhances model generalization without full fine-tuning.

The method effectively utilizes synthetic distorted images with confidence scoring.

Abstract

Image quality assessment (IQA) has long been a fundamental challenge in image understanding. In recent years, deep learning-based IQA methods have shown promising performance. However, the lack of large amounts of labeled data in the IQA field has hindered further advancements in these methods. This paper introduces DSMix, a novel data augmentation technique specifically designed for IQA tasks, aiming to overcome this limitation. DSMix leverages the distortion-induced sensitivity map (DSM) of an image as prior knowledge. It applies cut and mix operations to diverse categories of synthetic distorted images, assigning confidence scores to class labels based on the aforementioned prior knowledge. In the pre-training phase using DSMix-augmented data, knowledge distillation is employed to enhance the model's ability to extract semantic features. Experimental results on both synthetic and authentic IQA datasets demonstrate the significant predictive and generalization performance achieved by DSMix, without requiring fine-tuning of the full model. Code is available at \url{https://github.com/I2-Multimedia-Lab/DSMix}.