A Convolutional Neural Network-Based Low Complexity Filter

TL;DR

This paper introduces a low complexity CNN-based video filter using depth separable convolution and residual mapping, achieving effective artifact reduction with less computational cost and improved generalization.

Contribution

A novel low complexity CNN filter combining depth separable convolution, a residual mapping module, and a new weight initialization method for improved video artifact reduction.

Findings

Achieves 1.2% BD-rate reduction over H.265/HEVC

Reduces FLOPs by 79.1% compared to VR-CNN

Effective in generalization and adaptive filtering

Abstract

Convolutional Neural Network (CNN)-based filters have achieved significant performance in video artifacts reduction. However, the high complexity of existing methods makes it difficult to be applied in real usage. In this paper, a CNN-based low complexity filter is proposed. We utilize depth separable convolution (DSC) merged with the batch normalization (BN) as the backbone of our proposed CNN-based network. Besides, a weight initialization method is proposed to enhance the training performance. To solve the well known over smoothing problem for the inter frames, a frame-level residual mapping (RM) is presented. We analyze some of the mainstream methods like frame-level and block-level based filters quantitatively and build our CNN-based filter with frame-level control to avoid the extra complexity and artificial boundaries caused by block-level control. In addition, a novel module called RM is designed to restore the distortion from the learned residuals. As a result, we can effectively improve the generalization ability of the learning-based filter and reach an adaptive filtering effect. Moreover, this module is flexible and can be combined with other learning-based filters. The experimental results show that our proposed method achieves significant BD-rate reduction than H.265/HEVC. It achieves about 1.2% BD-rate reduction and 79.1% decrease in FLOPs than VR-CNN. Finally, the measurement on H.266/VVC and ablation studies are also conducted to ensure the effectiveness of the proposed method.