Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives

TL;DR

This paper introduces a new approach for assessing user-generated videos by considering both aesthetic and technical perspectives, supported by a large-scale human opinion dataset and achieving state-of-the-art results.

Contribution

It presents the DIVIDE-3k database and the DOVER/DOVER++ models that incorporate aesthetic and technical perspectives for improved video quality assessment.

Findings

Human opinions are influenced by both aesthetic and technical factors.

DOVER achieves state-of-the-art performance in UGC-VQA.

DOVER++ provides clear-cut quality evaluations from individual perspectives.

Abstract

The rapid increase in user-generated-content (UGC) videos calls for the development of effective video quality assessment (VQA) algorithms. However, the objective of the UGC-VQA problem is still ambiguous and can be viewed from two perspectives: the technical perspective, measuring the perception of distortions; and the aesthetic perspective, which relates to preference and recommendation on contents. To understand how these two perspectives affect overall subjective opinions in UGC-VQA, we conduct a large-scale subjective study to collect human quality opinions on overall quality of videos as well as perceptions from aesthetic and technical perspectives. The collected Disentangled Video Quality Database (DIVIDE-3k) confirms that human quality opinions on UGC videos are universally and inevitably affected by both aesthetic and technical perspectives. In light of this, we propose the Disentangled Objective Video Quality Evaluator (DOVER) to learn the quality of UGC videos based on the two perspectives. The DOVER proves state-of-the-art performance in UGC-VQA under very high efficiency. With perspective opinions in DIVIDE-3k, we further propose DOVER++, the first approach to provide reliable clear-cut quality evaluations from a single aesthetic or technical perspective. Code at https://github.com/VQAssessment/DOVER.