Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC

TL;DR

This paper introduces a partition-aware and adaptive-switching neural network approach for post-processing in HEVC video coding, effectively reducing artifacts by utilizing encoder partition information and content-adaptive models.

Contribution

It proposes a novel partition-aware CNN leveraging CU size info and an adaptive-switching neural network with multiple CNNs for improved artifact reduction in HEVC.

Findings

Significant artifact reduction demonstrated on benchmark sequences

Partition-aware CNN outperforms existing methods

Adaptive-switching CNN further improves visual quality

Abstract

This paper addresses neural network based post-processing for the state-of-the-art video coding standard, High Efficiency Video Coding (HEVC). We first propose a partition-aware Convolution Neural Network (CNN) that utilizes the partition information produced by the encoder to assist in the post-processing. In contrast to existing CNN-based approaches, which only take the decoded frame as input, the proposed approach considers the coding unit (CU) size information and combines it with the distorted decoded frame such that the artifacts introduced by HEVC are efficiently reduced. We further introduce an adaptive-switching neural network (ASN) that consists of multiple independent CNNs to adaptively handle the variations in content and distortion within compressed-video frames, providing further reduction in visual artifacts. Additionally, an iterative training procedure is proposed to train these independent CNNs attentively on different local patch-wise classes. Experiments on benchmark sequences demonstrate the effectiveness of our partition-aware and adaptive-switching neural networks. The source code can be found at http://min.sjtu.edu.cn/lwydemo/HEVCpostprocessing.html.