High-Efficiency Lossy Image Coding Through Adaptive Neighborhood Information Aggregation

TL;DR

This paper introduces a novel learned lossy image coding method that leverages adaptive neighborhood information aggregation through a combined convolution and self-attention approach, achieving state-of-the-art compression and significantly faster decoding.

Contribution

It proposes the ICSA unit and MCM for content-adaptive transform and probability estimation, integrated within a VAE framework for improved compression efficiency.

Findings

Surpasses VVC Intra and other LIC methods in rate-distortion performance.

Achieves over 60 times faster decoding speed with comparable model size.

Provides publicly accessible code and models for reproducible research.

Abstract

Questing for learned lossy image coding (LIC) with superior compression performance and computation throughput is challenging. The vital factor behind it is how to intelligently explore Adaptive Neighborhood Information Aggregation (ANIA) in transform and entropy coding modules. To this end, Integrated Convolution and Self-Attention (ICSA) unit is first proposed to form a content-adaptive transform to characterize and embed neighborhood information dynamically of any input. Then a Multistage Context Model (MCM) is devised to progressively use available neighbors following a pre-arranged spatial-channel order for accurate probability estimation in parallel. ICSA and MCM are stacked under a Variational AutoEncoder (VAE) architecture to derive rate-distortion optimized compact representation of input image via end-to-end learning. Our method reports state-of-the-art compression performance surpassing the VVC Intra and other prevalent LIC approaches across Kodak, CLIC, and Tecnick datasets; More importantly, our method offers $>$60$\times$ decoding speedup using a comparable-size model when compared with the most popular LIC method. All materials are made publicly accessible at https://njuvision.github.io/TinyLIC for reproducible research.