Scaling-up Perceptual Video Quality Assessment

TL;DR

This paper introduces OmniVQA, a scalable framework for perceptual video quality assessment that leverages large, human-in-the-loop datasets and a novel training strategy to achieve state-of-the-art results.

Contribution

The paper presents OmniVQA, the largest MIDB for VQA, along with a fine-grained benchmark and a complementary training strategy to improve quality understanding and rating.

Findings

Achieved state-of-the-art performance in quality understanding.

Achieved state-of-the-art performance in quality rating.

Created the largest VQA multi-modal instruction database.

Abstract

The data scaling law has been shown to significantly enhance the performance of large multi-modal models (LMMs) across various downstream tasks. However, in the domain of perceptual video quality assessment (VQA), the potential of scaling law remains unprecedented due to the scarcity of labeled resources and the insufficient scale of datasets. To address this, we propose \textbf{OmniVQA}, an efficient framework designed to efficiently build high-quality, human-in-the-loop VQA multi-modal instruction databases (MIDBs). We then scale up to create \textbf{OmniVQA-Chat-400K}, the largest MIDB in the VQA field concurrently. Our focus is on the technical and aesthetic quality dimensions, with abundant in-context instruction data to provide fine-grained VQA knowledge. Additionally, we have built the \textbf{OmniVQA-MOS-20K} dataset to enhance the model's quantitative quality rating capabilities. We then introduce a \textbf{complementary} training strategy that effectively leverages the knowledge from datasets for quality understanding and quality rating tasks. Furthermore, we propose the \textbf{OmniVQA-FG (fine-grain)-Benchmark} to evaluate the fine-grained performance of the models. Our results demonstrate that our models achieve state-of-the-art performance in both quality understanding and rating tasks.