Prediction-Aware Quality Enhancement of VVC Using CNN

TL;DR

This paper introduces a CNN-based method that leverages coding information from VVC bitstreams to enhance the quality of compressed video frames, significantly reducing artifacts at various bitrates.

Contribution

It presents a novel approach that incorporates coding stream data into CNN training for improved post-decoding quality enhancement of VVC videos.

Findings

BD-rate improvement of about 1% for luminance

BD-rate improvement of about 6% for chrominance

Effective artifact reduction at both low and high bitrates

Abstract

The upcoming video coding standard, Versatile Video Coding (VVC), has shown great improvement compared to its predecessor, High Efficiency Video Coding (HEVC), in terms of bitrate saving. Despite its substantial performance, compressed videos might still suffer from quality degradation at low bitrates due to coding artifacts such as blockiness, blurriness and ringing. In this work, we exploit Convolutional Neural Networks (CNN) to enhance quality of VVC coded frames after decoding in order to reduce low bitrate artifacts. The main contribution of this work is the use of coding information from the compressed bitstream. More precisely, the prediction information of intra frames is used for training the network in addition to the reconstruction information. The proposed method is applied on both luminance and chrominance components of intra coded frames of VVC. Experiments on VVC Test Model (VTM) show that, both in low and high bitrates, the use of coding information can improve the BD-rate performance by about 1% and 6% for luma and chroma components, respectively.