Encoder-Decoder-Based Intra-Frame Block Partitioning Decision

TL;DR

This paper introduces a neural network-based method using CNN and Transformer to accelerate intra-frame block partitioning in video encoding, achieving significant time reduction with minimal performance loss.

Contribution

It presents a novel encoder-decoder neural network architecture that fully parallelizes intra-mode decision, significantly speeding up the process in video coding.

Findings

87.84% reduction in encoding time

8.09% decrease in coding performance

Effective acceleration with minimal quality loss

Abstract

The recursive intra-frame block partitioning decision process, a crucial component of the next-generation video coding standards, exerts significant influence over the encoding time. In this paper, we propose an encoder-decoder neural network (NN) to accelerate this process. Specifically, a CNN is utilized to compress the pixel data of the largest coding unit (LCU) into a fixed-length vector. Subsequently, a Transformer decoder is employed to transcribe the fixed-length vector into a variable-length vector, which represents the block partitioning outcomes of the encoding LCU. The vector transcription process adheres to the constraints imposed by the block partitioning algorithm. By fully parallelizing the NN prediction in the intra-mode decision, substantial time savings can be attained during the decision phase. The experimental results obtained from high-definition (HD) sequences coding demonstrate that this framework achieves a remarkable 87.84\% reduction in encoding time, with a relatively small loss (8.09\%) of coding performance compared to AVS3 HPM4.0.