UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment

TL;DR

This paper investigates how saliency maps affect deep feature-based quality assessment of user-generated content, finding that deep features alone often outperform saliency-augmented models in correlating with perceptual quality.

Contribution

It introduces saliency integration into deep feature-based UGC quality metrics and evaluates their impact, providing a benchmark dataset and code for future research.

Findings

Deep features achieve high correlation with perceptual quality.

Adding saliency maps does not always improve performance.

The study offers publicly available datasets and code for benchmarking.

Abstract

The volume of User Generated Content (UGC) has increased in recent years. The challenge with this type of content is assessing its quality. So far, the state-of-the-art metrics are not exhibiting a very high correlation with perceptual quality. In this paper, we explore state-of-the-art metrics that extract/combine natural scene statistics and deep neural network features. We experiment with these by introducing saliency maps to improve perceptibility. We train and test our models using public datasets, namely, YouTube-UGC and KoNViD-1k. Preliminary results indicate that high correlations are achieved by using only deep features while adding saliency is not always boosting the performance. Our results and code will be made publicly available to serve as a benchmark for the research community and can be found on our project page: https://github.com/xinyiW915/SPIE-2023-Supplementary.