Model Selection CNN-based VVC QualityEnhancement

TL;DR

This paper introduces a CNN-based post-processing method for VVC video streams that leverages prediction signals and a model selection strategy to improve artifact removal, achieving notable bitrate savings.

Contribution

It proposes a novel CNN-based post-processing approach with a model selection scheme that utilizes prediction signals for enhanced video quality in VVC coding.

Findings

Prediction-aware CNN improves quality by -1.3% BD-BR.

Model selection adds an extra -0.5% BD-BR gain.

Method outperforms previous artifact removal techniques.

Abstract

Artifact removal and filtering methods are inevitable parts of video coding. On one hand, new codecs and compression standards come with advanced in-loop filters and on the other hand, displays are equipped with high capacity processing units for post-treatment of decoded videos. This paper proposes a Convolutional Neural Network (CNN)-based post-processing algorithm for intra and inter frames of Versatile Video Coding (VVC) coded streams. Depending on the frame type, this method benefits from normative prediction signal by feeding it as an additional input along with reconstructed signal and a Quantization Parameter (QP)-map to the CNN. Moreover, an optional Model Selection (MS) strategy is adopted to pick the best trained model among available ones at the encoder side and signal it to the decoder side. This MS strategy is applicable at both frame level and block level. The experiments under the Random Access (RA) configuration of the VVC Test Model (VTM-10.0) show that the proposed prediction-aware algorithm can bring an additional BD-BR gain of -1.3% compared to the method without the prediction information. Furthermore, the proposed MS scheme brings -0.5% more BD-BR gain on top of the prediction-aware method.