RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content

TL;DR

RAPIQUE is a fast, accurate, and efficient no-reference video quality assessment model for user-generated content, combining scene statistics and deep features to outperform existing methods in speed and accuracy.

Contribution

This paper introduces RAPIQUE, the first general spatial-temporal bandpass statistics model for UGC video quality prediction that is both accurate and significantly faster than state-of-the-art models.

Findings

RAPIQUE achieves top performance on large-scale UGC datasets.

It operates with orders-of-magnitude faster runtime than existing models.

The model is suitable for real-time and low-latency applications.

Abstract

Blind or no-reference video quality assessment of user-generated content (UGC) has become a trending, challenging, heretofore unsolved problem. Accurate and efficient video quality predictors suitable for this content are thus in great demand to achieve more intelligent analysis and processing of UGC videos. Previous studies have shown that natural scene statistics and deep learning features are both sufficient to capture spatial distortions, which contribute to a significant aspect of UGC video quality issues. However, these models are either incapable or inefficient for predicting the quality of complex and diverse UGC videos in practical applications. Here we introduce an effective and efficient video quality model for UGC content, which we dub the Rapid and Accurate Video Quality Evaluator (RAPIQUE), which we show performs comparably to state-of-the-art (SOTA) models but with orders-of-magnitude faster runtime. RAPIQUE combines and leverages the advantages of both quality-aware scene statistics features and semantics-aware deep convolutional features, allowing us to design the first general and efficient spatial and temporal (space-time) bandpass statistics model for video quality modeling. Our experimental results on recent large-scale UGC video quality databases show that RAPIQUE delivers top performances on all the datasets at a considerably lower computational expense. We hope this work promotes and inspires further efforts towards practical modeling of video quality problems for potential real-time and low-latency applications. To promote public usage, an implementation of RAPIQUE has been made freely available online: \url{https://github.com/vztu/RAPIQUE}.