Gated Fusion Network for SAO Filter and Inter Frame Prediction in Versatile Video Coding

TL;DR

This paper introduces a deep learning-based filter for VVC that enhances video quality and reduces bit rate by effectively suppressing artifacts without additional signaling overhead.

Contribution

A novel multi-scale CNN model with variable filter size and deconvolution for intra-inter frame quality enhancement in VVC.

Findings

Outperforms baseline VVC in BD-BR and BD-PSNR metrics.

Achieves 3.762% average bit rate savings.

Effectively suppresses visual artifacts like ringing and blocking.

Abstract

To achieve higher coding efficiency, Versatile Video Coding (VVC) includes several novel components, but at the expense of increasing decoder computational complexity. These technologies at a low bit rate often create contouring and ringing effects on the reconstructed frames and introduce various blocking artifacts at block boundaries. To suppress those visual artifacts, the VVC framework supports four post-processing filter operations. The interoperation of these filters introduces extra signaling bits and eventually becomes overhead at higher resolution video processing. In this paper, a novel deep learning-based model is proposed for sample adaptive offset (SAO) nonlinear filtering operation and substantiated the merits of intra-inter frame quality enhancement. We introduced a variable filter size multi-scale CNN (MSCNN) to improve the denoising operation and incorporated strided deconvolution for further computation improvement. We demonstrated that our deconvolution model can effectively be trained by leveraging the high-frequency edge features learned in a parallel fashion using feature fusion and residual learning. The simulation results demonstrate that the proposed method outperforms the baseline VVC method in BD-BR, BD-PSNR measurements and achieves an average of 3.762 % bit rate saving on the standard video test sequences.