Neural Network based Inter bi-prediction Blending

TL;DR

This paper introduces a neural network-based method to enhance bi-prediction blending in video coding, achieving better compression efficiency with minimal complexity increase, validated on the VVC standard.

Contribution

A novel neural network approach for bi-prediction blending that improves coding efficiency with a small, efficient model and practical implementation strategies.

Findings

BD-rate improvement of -1.4% on VVC in random access mode

Neural network model with fewer than 10k parameters

Effective CPU-based implementation and quantization methods

Abstract

This paper presents a learning-based method to improve bi-prediction in video coding. In conventional video coding solutions, the motion compensation of blocks from already decoded reference pictures stands out as the principal tool used to predict the current frame. Especially, the bi-prediction, in which a block is obtained by averaging two different motion-compensated prediction blocks, significantly improves the final temporal prediction accuracy. In this context, we introduce a simple neural network that further improves the blending operation. A complexity balance, both in terms of network size and encoder mode selection, is carried out. Extensive tests on top of the recently standardized VVC codec are performed and show a BD-rate improvement of -1.4% in random access configuration for a network size of fewer than 10k parameters. We also propose a simple CPU-based implementation and direct network quantization to assess the complexity/gains tradeoff in a conventional codec framework.