Multitask Learning for VVC Quality Enhancement and Super-Resolution

TL;DR

This paper proposes a multitask deep learning approach to improve decoded VVC video quality by simultaneously performing artifact removal and super-resolution, reducing model complexity and enhancing performance.

Contribution

It introduces a shared neural network trained for both quality enhancement and super-resolution, leveraging multitask learning for efficient video post-processing.

Findings

Effective artifact mitigation and super-resolution with fewer parameters.

Outperforms traditional specialized models in quality enhancement.

Reduces redundancy by joint training for multiple degradation levels.

Abstract

The latest video coding standard, called versatile video coding (VVC), includes several novel and refined coding tools at different levels of the coding chain. These tools bring significant coding gains with respect to the previous standard, high efficiency video coding (HEVC). However, the encoder may still introduce visible coding artifacts, mainly caused by coding decisions applied to adjust the bitrate to the available bandwidth. Hence, pre and post-processing techniques are generally added to the coding pipeline to improve the quality of the decoded video. These methods have recently shown outstanding results compared to traditional approaches, thanks to the recent advances in deep learning. Generally, multiple neural networks are trained independently to perform different tasks, thus omitting to benefit from the redundancy that exists between the models. In this paper, we investigate a learning-based solution as a post-processing step to enhance the decoded VVC video quality. Our method relies on multitask learning to perform both quality enhancement and super-resolution using a single shared network optimized for multiple degradation levels. The proposed solution enables a good performance in both mitigating coding artifacts and super-resolution with fewer network parameters compared to traditional specialized architectures.