ViSTRA3: Video Coding with Deep Parameter Adaptation and Post Processing

TL;DR

ViSTRA3 is a deep learning-based video compression framework that adaptively optimizes video parameters before encoding and uses CNN-based post-processing to improve quality, outperforming standard VVC in tests.

Contribution

Introduces a novel deep learning framework that enhances video compression by adaptive parameter selection and CNN-based post-processing, achieving better compression efficiency.

Findings

Achieves 1.8% and 3.7% BD-rate savings on PSNR and VMAF.

Outperforms the original VVC VTM in standard tests.

Demonstrates effective integration with existing video coding standards.

Abstract

This paper presents a deep learning-based video compression framework (ViSTRA3). The proposed framework intelligently adapts video format parameters of the input video before encoding, subsequently employing a CNN at the decoder to restore their original format and enhance reconstruction quality. ViSTRA3 has been integrated with the H.266/VVC Test Model VTM 14.0, and evaluated under the Joint Video Exploration Team Common Test Conditions. Bj{\o}negaard Delta (BD) measurement results show that the proposed framework consistently outperforms the original VVC VTM, with average BD-rate savings of 1.8% and 3.7% based on the assessment of PSNR and VMAF.