MLIC++: Linear Complexity Multi-Reference Entropy Modeling for Learned Image Compression

TL;DR

This paper introduces MLIC++, a learned image compression method that employs a novel entropy model with linear complexity to efficiently capture multi-scale correlations, achieving state-of-the-art results especially on high-resolution images.

Contribution

The paper proposes MEM++, a multi-reference entropy model with linear complexity for high-resolution image compression, enabling efficient global and local context modeling within a single framework.

Findings

Achieves 13.39% BD-rate reduction on Kodak dataset.

Maintains linear computational complexity with resolution.

Outperforms previous methods in PSNR performance.

Abstract

The latent representation in learned image compression encompasses channel-wise, local spatial, and global spatial correlations, which are essential for the entropy model to capture for conditional entropy minimization. Efficiently capturing these contexts within a single entropy model, especially in high-resolution image coding, presents a challenge due to the computational complexity of existing global context modules. To address this challenge, we propose the Linear Complexity Multi-Reference Entropy Model (MEM$^{++}$). Specifically, the latent representation is partitioned into multiple slices. For channel-wise contexts, previously compressed slices serve as the context for compressing a particular slice. For local contexts, we introduce a shifted-window-based checkerboard attention module. This module ensures linear complexity without sacrificing performance. For global contexts, we propose a linear complexity attention mechanism. It captures global correlations by decomposing the softmax operation, enabling the implicit computation of attention maps from previously decoded slices. Using MEM$^{++}$ as the entropy model, we develop the image compression method MLIC$^{++}$. Extensive experimental results demonstrate that MLIC$^{++}$ achieves state-of-the-art performance, reducing BD-rate by $13.39\%$ on the Kodak dataset compared to VTM-17.0 in Peak Signal-to-Noise Ratio (PSNR). Furthermore, MLIC$^{++}$ exhibits linear computational complexity and memory consumption with resolution, making it highly suitable for high-resolution image coding. Code and pre-trained models are available at https://github.com/JiangWeibeta/MLIC. Training dataset is available at https://huggingface.co/datasets/Whiteboat/MLIC-Train-100K.