LEHA-CVQAD: Dataset To Enable Generalized Video Quality Assessment of Compression Artifacts

TL;DR

The paper introduces LEHA-CVQAD, a large-scale dataset for video quality assessment of compression artifacts, along with a new metric RDAE to evaluate VQA models' effectiveness in bitrate-quality ordering.

Contribution

It provides a comprehensive dataset with diverse compressed videos and a novel RDAE metric for better evaluation of VQA models in compression scenarios.

Findings

Existing VQA metrics show high RDAE and low correlation on the dataset.

LEHA-CVQAD enables better understanding of compression artifact perception.

The dataset supports codec parameter tuning and blind evaluation.

Abstract

We propose the LEHA-CVQAD (Large-scale Enriched Human-Annotated Compressed Video Quality Assessment) dataset, which comprises 6,240 clips for compression-oriented video quality assessment. 59 source videos are encoded with 186 codec-preset variants, 1.8M pairwise, and 1.5k MOS ratings are fused into a single quality scale; part of the videos remains hidden for blind evaluation. We also propose Rate-Distortion Alignment Error (RDAE), a novel evaluation metric that quantifies how well VQA models preserve bitrate-quality ordering, directly supporting codec parameter tuning. Testing IQA/VQA methods reveals that popular VQA metrics exhibit high RDAE and lower correlations, underscoring the dataset challenges and utility. The open part and the results of LEHA-CVQAD are available at https://aleksandrgushchin.github.io/lcvqad/