Efficient Adaptation of Neural Network Filter for Video Compression

TL;DR

This paper introduces a fast finetuning method for neural network filters used in video postprocessing, which adapts to specific content at the encoder side, improving compression efficiency with minimal bitrate increase.

Contribution

The paper proposes finetuning only convolutional biases for neural filters, enabling rapid adaptation and easy integration into existing video codecs.

Findings

Achieves up to 9.7% BD-rate reduction on test sequences.

Faster convergence compared to traditional finetuning methods.

Applicable to real-world video coding scenarios.

Abstract

We present an efficient finetuning methodology for neural-network filters which are applied as a postprocessing artifact-removal step in video coding pipelines. The fine-tuning is performed at encoder side to adapt the neural network to the specific content that is being encoded. In order to maximize the PSNR gain and minimize the bitrate overhead, we propose to finetune only the convolutional layers' biases. The proposed method achieves convergence much faster than conventional finetuning approaches, making it suitable for practical applications. The weight-update can be included into the video bitstream generated by the existing video codecs. We show that our method achieves up to 9.7% average BD-rate gain when compared to the state-of-art Versatile Video Coding (VVC) standard codec on 7 test sequences.