Deep Learning based Full-reference and No-reference Quality Assessment Models for Compressed UGC Videos

TL;DR

This paper introduces a deep learning framework for assessing the quality of compressed user-generated videos, combining feature fusion from CNN layers with a temporal pooling strategy for accurate quality prediction.

Contribution

It presents a novel deep learning VQA model that fuses intermediate CNN features for both full-reference and no-reference quality assessment of compressed UGC videos.

Findings

Achieves state-of-the-art performance on compressed UGC VQA database

Performs well on in-the-wild UGC VQA datasets

Effectively combines feature fusion and temporal pooling techniques

Abstract

In this paper, we propose a deep learning based video quality assessment (VQA) framework to evaluate the quality of the compressed user's generated content (UGC) videos. The proposed VQA framework consists of three modules, the feature extraction module, the quality regression module, and the quality pooling module. For the feature extraction module, we fuse the features from intermediate layers of the convolutional neural network (CNN) network into final quality-aware feature representation, which enables the model to make full use of visual information from low-level to high-level. Specifically, the structure and texture similarities of feature maps extracted from all intermediate layers are calculated as the feature representation for the full reference (FR) VQA model, and the global mean and standard deviation of the final feature maps fused by intermediate feature maps are calculated as the feature representation for the no reference (NR) VQA model. For the quality regression module, we use the fully connected (FC) layer to regress the quality-aware features into frame-level scores. Finally, a subjectively-inspired temporal pooling strategy is adopted to pool frame-level scores into the video-level score. The proposed model achieves the best performance among the state-of-the-art FR and NR VQA models on the Compressed UGC VQA database and also achieves pretty good performance on the in-the-wild UGC VQA databases.