MFRNet: A New CNN Architecture for Post-Processing and In-loop Filtering

TL;DR

This paper introduces MFRNet, a novel CNN architecture designed to enhance video compression quality through post-processing and in-loop filtering, demonstrating significant coding gains over existing methods.

Contribution

The paper presents MFRNet, a new CNN architecture with multi-level feature review residual dense blocks, integrated into HEVC and VVC, achieving improved coding efficiency.

Findings

Up to 21% BD-rate VMAF gain in PP with VVC.

Up to 16% BD-rate VMAF gain in ILF with HEVC.

Outperforms existing CNN-based approaches in video coding tasks.

Abstract

In this paper, we propose a novel convolutional neural network (CNN) architecture, MFRNet, for post-processing (PP) and in-loop filtering (ILF) in the context of video compression. This network consists of four Multi-level Feature review Residual dense Blocks (MFRBs), which are connected using a cascading structure. Each MFRB extracts features from multiple convolutional layers using dense connections and a multi-level residual learning structure. In order to further improve information flow between these blocks, each of them also reuses high dimensional features from the previous MFRB. This network has been integrated into PP and ILF coding modules for both HEVC (HM 16.20) and VVC (VTM 7.0), and fully evaluated under the JVET Common Test Conditions using the Random Access configuration. The experimental results show significant and consistent coding gains over both anchor codecs (HEVC HM and VVC VTM) and also over other existing CNN-based PP/ILF approaches based on Bjontegaard Delta measurements using both PSNR and VMAF for quality assessment. When MFRNet is integrated into HM 16.20, gains up to 16.0% (BD-rate VMAF) are demonstrated for ILF, and up to 21.0% (BD-rate VMAF) for PP. The respective gains for VTM 7.0 are up to 5.1% for ILF and up to 7.1% for PP.