Full Reference Video Quality Assessment for Machine Learning-Based Video Codecs

TL;DR

This paper introduces a new dataset and a highly accurate full reference video quality assessment model tailored for machine learning-based video codecs, addressing the limitations of existing metrics.

Contribution

It provides the first dataset specifically labeled for ML video codecs and proposes a novel FRVQA model with near-perfect correlation to subjective quality assessments.

Findings

The new dataset enables better evaluation of ML video codecs.

The proposed FRVQA model achieves PCC and SRCC of 0.99.

Open source release facilitates further research and improvements.

Abstract

Machine learning-based video codecs have made significant progress in the past few years. A critical area in the development of ML-based video codecs is an accurate evaluation metric that does not require an expensive and slow subjective test. We show that existing evaluation metrics that were designed and trained on DSP-based video codecs are not highly correlated to subjective opinion when used with ML video codecs due to the video artifacts being quite different between ML and video codecs. We provide a new dataset of ML video codec videos that have been accurately labeled for quality. We also propose a new full reference video quality assessment (FRVQA) model that achieves a Pearson Correlation Coefficient (PCC) of 0.99 and a Spearman's Rank Correlation Coefficient (SRCC) of 0.99 at the model level. We make the dataset and FRVQA model open source to help accelerate research in ML video codecs, and so that others can further improve the FRVQA model.