DCVQE: A Hierarchical Transformer for Video Quality Assessment

TL;DR

DCVQE introduces a hierarchical Transformer-based approach for no-reference video quality assessment, effectively capturing multi-level quality features and improving robustness through a novel correlation loss.

Contribution

The paper proposes a novel hierarchical Transformer architecture (DCVQE) with a divide-and-conquer strategy for improved NR-VQA performance.

Findings

Outperforms existing methods on multiple datasets

Demonstrates robustness and accuracy in video quality prediction

Effectively models hierarchical quality relationships

Abstract

The explosion of user-generated videos stimulates a great demand for no-reference video quality assessment (NR-VQA). Inspired by our observation on the actions of human annotation, we put forward a Divide and Conquer Video Quality Estimator (DCVQE) for NR-VQA. Starting from extracting the frame-level quality embeddings (QE), our proposal splits the whole sequence into a number of clips and applies Transformers to learn the clip-level QE and update the frame-level QE simultaneously; another Transformer is introduced to combine the clip-level QE to generate the video-level QE. We call this hierarchical combination of Transformers as a Divide and Conquer Transformer (DCTr) layer. An accurate video quality feature extraction can be achieved by repeating the process of this DCTr layer several times. Taking the order relationship among the annotated data into account, we also propose a novel correlation loss term for model training. Experiments on various datasets confirm the effectiveness and robustness of our DCVQE model.