Analytic Score Optimization for Multi Dimension Video Quality Assessment

TL;DR

This paper introduces UltraVQA, a large multi-dimensional video quality dataset with detailed annotations, and proposes Analytic Score Optimization (ASO), a novel method that improves multi-dimensional quality assessment by aligning with human judgments.

Contribution

The paper presents UltraVQA, a comprehensive multi-dimensional VQA dataset, and introduces ASO, a theoretically grounded optimization method that enhances quality score prediction accuracy.

Findings

ASO outperforms baseline models and APIs in quality prediction.

UltraVQA provides rich, multi-dimensional annotations with human rationale.

The method reduces mean absolute error in video quality assessment.

Abstract

Video Quality Assessment (VQA) is evolving beyond single-number mean opinion score toward richer, multi-faceted evaluations of video content. In this paper, we present a large-scale multi-dimensional VQA dataset UltraVQA that encompasses diverse User-Generated Content~(UGC) annotated across five key quality dimensions: Motion Quality, Motion Amplitude, Aesthetic Quality, Content Quality, and Clarity Quality. Each video in our dataset is scored by over 3 human raters on these dimensions, with fine-grained sub-attribute labels, and accompanied by an explanatory rationale generated by GPT based on the collective human judgments. To better leverage these rich annotations and improve discrete quality score assessment, we introduce Analytic Score Optimization (ASO), a theoretically grounded post-training objective derived for multi-dimensional VQA. By reframing quality assessment as a regularized decision-making process, we obtain a closed-form solution that naturally captures the ordinal nature of human ratings, ensuring alignment with human ranking preferences. In experiments, our method outperforms most baselines including closed-source APIs and open-source models, while also reducing mean absolute error (MAE) in quality prediction. Our work highlights the importance of multi-dimensional, interpretable annotations and reinforcement-based alignment in advancing video quality assessment.