Neural Generation of Blocks for Video Coding

TL;DR

This paper introduces a method to encode content-specific neural network parameters within video streams to improve compression efficiency for certain types of visual content, such as panning or detailed shapes.

Contribution

It proposes encoding learned neural network parameters in the bitstream and selectively generating content for blocks where this method outperforms traditional prediction techniques.

Findings

Improved compression for panning and zooming scenes.

Effective use of neural networks for content-specific video generation.

Selective generation enhances overall compression efficiency.

Abstract

Well-trained generative neural networks (GNN) are very efficient at compressing visual information for static images in their learned parameters but not as efficient as inter- and intra-prediction for most video content. However, for content entering a frame, such as during panning or zooming out, and content with curves, irregular shapes, or fine detail, generation by a GNN can give better compression efficiency (lower rate-distortion). This paper proposes encoding content-specific learned parameters of a GNN within a video bitstream at specific times and using the GNN to generate content for specific ranges of blocks and frames. The blocks to generate are just the ones for which generation gives more efficient compression than inter- or intra- prediction. This approach maximizes the usefulness of the information contained in the learned parameters.