Content Adaptive Latents and Decoder for Neural Image Compression

TL;DR

This paper introduces a neural image compression framework that adaptively optimizes both the latent representations and decoder features, leading to state-of-the-art compression performance by reducing redundancy and enhancing content adaptability.

Contribution

It proposes novel content adaptive methods for both latents and decoder, specifically channel dropping and feature transformation, to improve neural image compression adaptability.

Findings

Achieves state-of-the-art compression performance.

Reduces redundancy in latents via channel dropping.

Enhances decoder adaptability with feature transformation.

Abstract

In recent years, neural image compression (NIC) algorithms have shown powerful coding performance. However, most of them are not adaptive to the image content. Although several content adaptive methods have been proposed by updating the encoder-side components, the adaptability of both latents and the decoder is not well exploited. In this work, we propose a new NIC framework that improves the content adaptability on both latents and the decoder. Specifically, to remove redundancy in the latents, our content adaptive channel dropping (CACD) method automatically selects the optimal quality levels for the latents spatially and drops the redundant channels. Additionally, we propose the content adaptive feature transformation (CAFT) method to improve decoder-side content adaptability by extracting the characteristic information of the image content, which is then used to transform the features in the decoder side. Experimental results demonstrate that our proposed methods with the encoder-side updating algorithm achieve the state-of-the-art performance.