On Versatile Video Coding at UHD with Machine-Learning-Based Super-Resolution

TL;DR

This paper introduces a novel 4K video coding approach combining VVC with machine learning super-resolution, achieving significant bitrate savings and improved visual quality at low bitrates.

Contribution

It presents a new coding chain that integrates spatial down- and upscaling with VVC and super-resolution, demonstrating improved efficiency for 4K video compression.

Findings

Up to 12% BDR rate savings at high QP values.

Up to 18% BDR rate savings at higher QP values.

Enhanced detail preservation and reduced artifacts.

Abstract

Coding 4K data has become of vital interest in recent years, since the amount of 4K data is significantly increasing. We propose a coding chain with spatial down- and upscaling that combines the next-generation VVC codec with machine learning based single image super-resolution algorithms for 4K. The investigated coding chain, which spatially downscales the 4K data before coding, shows superior quality than the conventional VVC reference software for low bitrate scenarios. Throughout several tests, we find that up to 12 % and 18 % Bjontegaard delta rate gains can be achieved on average when coding 4K sequences with VVC and QP values above 34 and 42, respectively. Additionally, the investigated scenario with up- and downscaling helps to reduce the loss of details and compression artifacts, as it is shown in a visual example.