Q-Save: Towards Scoring and Attribution for Generated Video Evaluation

TL;DR

Q-Save introduces a comprehensive dataset and a unified model for evaluating AI-generated videos, addressing previous limitations by jointly assessing quality dimensions and providing interpretability.

Contribution

It presents a new dataset with detailed annotations and a novel evaluation model that jointly scores and attributes quality in generated videos.

Findings

Q-Save outperforms existing methods in quality prediction accuracy.

The model provides interpretable attributions for video quality.

The dataset enables systematic evaluation of multiple quality dimensions.

Abstract

Evaluating AI-generated video (AIGV) quality hinges on three crucial dimensions: visual quality, dynamic quality, and text-video alignment. While numerous evaluation datasets and algorithms have been proposed, existing approaches are constrained by two limitations: the absence of systematic definitions for evaluation dimensions, and the isolated treatment of the three dimensions in separate models. Therefore, we introduce Q-Save, a holistic benchmark dataset and unified evaluation model for AIGV quality assessment. The Q-Save dataset contains nearly 10,000 video samples, each annotated with Mean Opinion Scores (MOS) and fine-grained attribution explanations across the three core dimensions. Leveraging this attribution-annotated dataset, we train the proposed Q-Save model, which adopts the SlowFast framework to balance accuracy and efficiency, and employs a three-stage training strategy with Chain-of-Thought (COT) formatted data: Supervised Fine-Tuning (SFT), Grouped Relative Policy Optimization (GRPO), and a final SFT round for stability, to jointly perform quality scoring and attribution generation. Experimental results demonstrate that Q-Save achieves superior performance in AIGV quality prediction while providing interpretable justifications. Code and dataset will be released upon publication.