Saliency-Aware Spatio-Temporal Artifact Detection for Compressed Video Quality Assessment

TL;DR

This paper introduces SSTAM, a new quality assessment metric for compressed videos that detects spatial and temporal artifacts using saliency models and improved self-attention, outperforming existing metrics.

Contribution

The paper presents a novel saliency-aware spatio-temporal artifact detection method and a quality metric for compressed videos, addressing limitations of prior approaches.

Findings

SSTAM outperforms state-of-the-art quality metrics.

The saliency model effectively detects spatial PEAs with low computational cost.

Enhanced TimeSFormer improves temporal artifact detection.

Abstract

Compressed videos often exhibit visually annoying artifacts, known as Perceivable Encoding Artifacts (PEAs), which dramatically degrade video visual quality. Subjective and objective measures capable of identifying and quantifying various types of PEAs are critical in improving visual quality. In this paper, we investigate the influence of four spatial PEAs (i.e. blurring, blocking, bleeding, and ringing) and two temporal PEAs (i.e. flickering and floating) on video quality. For spatial artifacts, we propose a visual saliency model with a low computational cost and higher consistency with human visual perception. In terms of temporal artifacts, self-attention based TimeSFormer is improved to detect temporal artifacts. Based on the six types of PEAs, a quality metric called Saliency-Aware Spatio-Temporal Artifacts Measurement (SSTAM) is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art metrics. We believe that SSTAM will be beneficial for optimizing video coding techniques.