Combined neural network-based intra prediction and transform selection

TL;DR

This paper introduces a combined neural network-based intra prediction and transform selection method for block-based video codecs, achieving significant rate-distortion efficiency improvements.

Contribution

It presents a novel integrated approach for neural network-based intra prediction and transform selection within a video codec, enhancing compression performance.

Findings

Achieved up to 3.71% BD-rate reduction in all-intra mode.

Integrated neural network prediction improves codec efficiency.

Demonstrated effectiveness within VTM-8.0 framework.

Abstract

The interactions between different tools added successively to a block-based video codec are critical to its rate-distortion efficiency. In particular, when deep neural network-based intra prediction modes are inserted into a block-based video codec, as the neural network-based prediction function cannot be easily characterized, the adaptation of the transform selection process to the new modes can hardly be performed manually. That is why this paper presents a combined neural network-based intra prediction and transform selection for a block-based video codec. When putting a single neural network-based intra prediction mode and the learned prediction of the selected LFNST pair index into VTM-8.0, -3.71%, -3.17%, and -3.37% of mean BD-rate reduction in all-intra is obtained.