Enhancing VVC with Deep Learning based Multi-Frame Post-Processing

TL;DR

This paper introduces a CNN-based multi-frame post-processing method using CVEGAN to improve visual quality in VVC, demonstrating consistent gains in PSNR on CLIC 2022 sequences.

Contribution

It presents a novel deep learning-based multi-frame post-processing approach integrated with VVC to enhance reconstructed video quality.

Findings

Achieved consistent PSNR improvements over original VVC VTM at same bitrates.

Successfully integrated the method into VVC and submitted to CLIC 2022.

Demonstrated perceptually-inspired GAN architecture effectiveness.

Abstract

This paper describes a CNN-based multi-frame post-processing approach based on a perceptually-inspired Generative Adversarial Network architecture, CVEGAN. This method has been integrated with the Versatile Video Coding Test Model (VTM) 15.2 to enhance the visual quality of the final reconstructed content. The evaluation results on the CLIC 2022 validation sequences show consistent coding gains over the original VVC VTM at the same bitrates when assessed by PSNR. The integrated codec has been submitted to the Challenge on Learned Image Compression (CLIC) 2022 (video track), and the team name associated with this submission is BVI_VC.