MDS-VQA: Model-Informed Data Selection for Video Quality Assessment

TL;DR

This paper introduces MDS-VQA, a data selection method that identifies challenging and diverse videos to improve video quality assessment models through targeted fine-tuning, enhancing their accuracy and generalization.

Contribution

MDS-VQA is a novel model-informed data selection approach that combines difficulty estimation and diversity measurement to optimize unlabeled data curation for VQA model improvement.

Findings

Selected 5% of data improves SRCC from 0.651 to 0.722

Method outperforms baseline data selection strategies

Achieves top gMAD rank indicating strong generalization

Abstract

Learning-based video quality assessment (VQA) has advanced rapidly, yet progress is increasingly constrained by a disconnect between model design and dataset curation. Model-centric approaches often iterate on fixed benchmarks, while data-centric efforts collect new human labels without systematically targeting the weaknesses of existing VQA models. Here, we describe MDS-VQA, a model-informed data selection mechanism for curating unlabeled videos that are both difficult for the base VQA model and diverse in content. Difficulty is estimated by a failure predictor trained with a ranking objective, and diversity is measured using deep semantic video features, with a greedy procedure balancing the two under a constrained labeling budget. Experiments across multiple VQA datasets and models demonstrate that MDS-VQA identifies diverse, challenging samples that are particularly informative for active fine-tuning. With only a 5% selected subset per target domain, the fine-tuned model improves mean SRCC from 0.651 to 0.722 and achieves the top gMAD rank, indicating strong adaptation and generalization.