Multi-Context Dual Hyper-Prior Neural Image Compression

TL;DR

This paper introduces a Transformer-based nonlinear transform and a dual hyper-prior entropy model with global context for neural image compression, significantly improving rate-distortion performance by capturing long-range dependencies.

Contribution

It presents a novel Transformer-based transform and a dual hyper-prior entropy model with global context, addressing limitations of convolutional transforms in capturing long-range dependencies.

Findings

Outperforms state-of-the-art methods in rate-distortion performance

Effectively models long-range dependencies in images

Improves decorrelation of latent representations

Abstract

Transform and entropy models are the two core components in deep image compression neural networks. Most existing learning-based image compression methods utilize convolutional-based transform, which lacks the ability to model long-range dependencies, primarily due to the limited receptive field of the convolution operation. To address this limitation, we propose a Transformer-based nonlinear transform. This transform has the remarkable ability to efficiently capture both local and global information from the input image, leading to a more decorrelated latent representation. In addition, we introduce a novel entropy model that incorporates two different hyperpriors to model cross-channel and spatial dependencies of the latent representation. To further improve the entropy model, we add a global context that leverages distant relationships to predict the current latent more accurately. This global context employs a causal attention mechanism to extract long-range information in a content-dependent manner. Our experiments show that our proposed framework performs better than the state-of-the-art methods in terms of rate-distortion performance.