Content Adaptive Optimization for Neural Image Compression

TL;DR

This paper introduces a content adaptive optimization method for neural image compression that improves rate-distortion performance by iteratively adapting the latent representation to specific content without changing the model parameters.

Contribution

It proposes an iterative content adaptive optimization procedure that enhances neural image compression performance across architectures without additional parameter transmission.

Findings

Adaptive optimization increases rate-distortion performance.

Method effectively adapts pretrained models to different content.

Performance approaches models trained specifically for new content.

Abstract

The field of neural image compression has witnessed exciting progress as recently proposed architectures already surpass the established transform coding based approaches. While, so far, research has mainly focused on architecture and model improvements, in this work we explore content adaptive optimization. To this end, we introduce an iterative procedure which adapts the latent representation to the specific content we wish to compress while keeping the parameters of the network and the predictive model fixed. Our experiments show that this allows for an overall increase in rate-distortion performance, independently of the specific architecture used. Furthermore, we also evaluate this strategy in the context of adapting a pretrained network to other content that is different in visual appearance or resolution. Here, our experiments show that our adaptation strategy can largely close the gap as compared to models specifically trained for the given content while having the benefit that no additional data in the form of model parameter updates has to be transmitted.