Active Fine-Tuning from gMAD Examples Improves Blind Image Quality Assessment

TL;DR

This paper introduces an active fine-tuning approach for blind image quality assessment (BIQA) that uses gMAD examples to identify weaknesses, enabling iterative improvement and better generalization of the model across diverse datasets.

Contribution

The paper proposes a novel active learning framework that leverages gMAD examples for iterative fine-tuning of BIQA models, enhancing their robustness and generalization capabilities.

Findings

Fine-tuning with gMAD examples improves model generalization.

Active learning from gMAD examples enhances BIQA performance.

The method maintains performance on existing datasets while improving on new data.

Abstract

The research in image quality assessment (IQA) has a long history, and significant progress has been made by leveraging recent advances in deep neural networks (DNNs). Despite high correlation numbers on existing IQA datasets, DNN-based models may be easily falsified in the group maximum differentiation (gMAD) competition with strong counterexamples being identified. Here we show that gMAD examples can be used to improve blind IQA (BIQA) methods. Specifically, we first pre-train a DNN-based BIQA model using multiple noisy annotators, and fine-tune it on multiple subject-rated databases of synthetically distorted images, resulting in a top-performing baseline model. We then seek pairs of images by comparing the baseline model with a set of full-reference IQA methods in gMAD. The resulting gMAD examples are most likely to reveal the relative weaknesses of the baseline, and suggest potential ways for refinement. We query ground truth quality annotations for the selected images in a well controlled laboratory environment, and further fine-tune the baseline on the combination of human-rated images from gMAD and existing databases. This process may be iterated, enabling active and progressive fine-tuning from gMAD examples for BIQA. We demonstrate the feasibility of our active learning scheme on a large-scale unlabeled image set, and show that the fine-tuned method achieves improved generalizability in gMAD, without destroying performance on previously trained databases.