Multi-Scale Feature Prediction with Auxiliary-Info for Neural Image Compression

TL;DR

This paper introduces a multi-scale feature prediction framework with auxiliary information for neural image compression, significantly improving rate-distortion performance by leveraging auxiliary networks and modules for better feature and residual prediction.

Contribution

The paper proposes a novel auxiliary-info-guided multi-scale feature prediction structure with new modules, enhancing neural image compression performance over existing models.

Findings

Achieves 19.49% higher rate-distortion performance than VVC on Tecnick dataset.

Outperforms other neural image compression models in extensive experiments.

Demonstrates effectiveness of proposed modules through ablation studies.

Abstract

Recently, significant improvements in rate-distortion performance of image compression have been achieved with deep-learning techniques. A key factor in this success is the use of additional bits to predict an approximation of the latent vector, which is the output of the encoder, through another neural network. Then, only the difference between the prediction and the latent vector is coded into the bitstream, along with its estimated probability distribution. We introduce a new predictive structure consisting of the auxiliary coarse network and the main network, inspired by neural video compression. The auxiliary coarse network encodes the auxiliary information and predicts the approximation of the original image as multi-scale features. The main network encodes the residual between the predicted feature from the auxiliary coarse network and the feature of the original image. To further leverage our new structure, we propose Auxiliary info-guided Feature Prediction (AFP) module that uses global correlation to predict more accurate predicted features. Moreover, we present Context Junction module that refines the auxiliary feature from AFP module and produces the residuals between the refined features and the original image features. Finally, we introduce Auxiliary info-guided Parameter Estimation (APE) module, which predicts the approximation of the latent vector and estimates the probability distribution of these residuals. We demonstrate the effectiveness of the proposed modules by various ablation studies. Under extensive experiments, our model outperforms other neural image compression models and achieves a 19.49\% higher rate-distortion performance than VVC on Tecnick dataset.