Complexity Reduction of Learned In-Loop Filtering in Video Coding

TL;DR

This paper introduces a novel sparsity and structured pruning approach to reduce the computational complexity of learned in-loop filters in video coding, maintaining performance while enabling practical deployment.

Contribution

It proposes a three-step training process involving magnitude-guided pruning, neuron removal, and fine-tuning to effectively reduce complexity of neural network-based in-loop filters.

Findings

Network parameters can be significantly reduced.

Minimal impact on network performance after pruning.

Enhanced practicality of learned in-loop filters.

Abstract

In video coding, in-loop filters are applied on reconstructed video frames to enhance their perceptual quality, before storing the frames for output. Conventional in-loop filters are obtained by hand-crafted methods. Recently, learned filters based on convolutional neural networks that utilize attention mechanisms have been shown to improve upon traditional techniques. However, these solutions are typically significantly more computationally expensive, limiting their potential for practical applications. The proposed method uses a novel combination of sparsity and structured pruning for complexity reduction of learned in-loop filters. This is done through a three-step training process of magnitude-guidedweight pruning, insignificant neuron identification and removal, and fine-tuning. Through initial tests we find that network parameters can be significantly reduced with a minimal impact on network performance.