Multi-rate adaptive transform coding for video compression

TL;DR

This paper introduces a multi-rate adaptive transform coding framework that enhances traditional video compression by integrating learned transforms and entropy coding, achieving better quality and efficiency.

Contribution

It proposes a flexible, learned transform coding approach that can replace or improve existing linear transforms in video codecs, operating across multiple rate-distortion points.

Findings

Substantial BD-rate reductions achieved.

Improved perceptual quality over nonlinear transforms.

Lower computational cost compared to complex models.

Abstract

Contemporary lossy image and video coding standards rely on transform coding, the process through which pixels are mapped to an alternative representation to facilitate efficient data compression. Despite impressive performance of end-to-end optimized compression with deep neural networks, the high computational and space demands of these models has prevented them from superseding the relatively simple transform coding found in conventional video codecs. In this study, we propose learned transforms and entropy coding that may either serve as (non)linear drop-in replacements, or enhancements for linear transforms in existing codecs. These transforms can be multi-rate, allowing a single model to operate along the entire rate-distortion curve. To demonstrate the utility of our framework, we augmented the DCT with learned quantization matrices and adaptive entropy coding to compress intra-frame AV1 block prediction residuals. We report substantial BD-rate and perceptual quality improvements over more complex nonlinear transforms at a fraction of the computational cost.