CALLIC: Content Adaptive Learning for Lossless Image Compression

TL;DR

CALLIC introduces a content-adaptive, efficient method for lossless image compression by combining the MDL principle with PETL, achieving state-of-the-art results through a novel self-attention mechanism and progressive fine-tuning.

Contribution

The paper proposes a novel content-adaptive approach called CALLIC that leverages PETL and a new self-attention mechanism for improved lossless image compression.

Findings

Sets new state-of-the-art performance on multiple datasets.

Reduces encoding time through Cache then Crop Inference (CCI).

Enhances compression efficiency with Rate-guided Progressive Fine-Tuning (RPFT).

Abstract

Learned lossless image compression has achieved significant advancements in recent years. However, existing methods often rely on training amortized generative models on massive datasets, resulting in sub-optimal probability distribution estimation for specific testing images during encoding process. To address this challenge, we explore the connection between the Minimum Description Length (MDL) principle and Parameter-Efficient Transfer Learning (PETL), leading to the development of a novel content-adaptive approach for learned lossless image compression, dubbed CALLIC. Specifically, we first propose a content-aware autoregressive self-attention mechanism by leveraging convolutional gating operations, termed Masked Gated ConvFormer (MGCF), and pretrain MGCF on training dataset. Cache then Crop Inference (CCI) is proposed to accelerate the coding process. During encoding, we decompose pre-trained layers, including depth-wise convolutions, using low-rank matrices and then adapt the incremental weights on testing image by Rate-guided Progressive Fine-Tuning (RPFT). RPFT fine-tunes with gradually increasing patches that are sorted in descending order by estimated entropy, optimizing learning process and reducing adaptation time. Extensive experiments across diverse datasets demonstrate that CALLIC sets a new state-of-the-art (SOTA) for learned lossless image compression.