Learning Convolutional Networks for Content-weighted Image Compression

TL;DR

This paper introduces a CNN-based image compression method that adaptively allocates bits based on local content, using a content-weighted importance map to improve compression quality and outperform traditional standards like JPEG.

Contribution

The paper proposes a novel content-aware bit rate allocation scheme using importance maps, enabling end-to-end training of a CNN for image compression without discrete entropy estimation.

Findings

Outperforms JPEG and JPEG 2000 in SSIM index at low bit rates.

Produces sharper edges, richer textures, and fewer artifacts.

Enables end-to-end optimization of the compression network.

Abstract

Lossy image compression is generally formulated as a joint rate-distortion optimization to learn encoder, quantizer, and decoder. However, the quantizer is non-differentiable, and discrete entropy estimation usually is required for rate control. These make it very challenging to develop a convolutional network (CNN)-based image compression system. In this paper, motivated by that the local information content is spatially variant in an image, we suggest that the bit rate of the different parts of the image should be adapted to local content. And the content aware bit rate is allocated under the guidance of a content-weighted importance map. Thus, the sum of the importance map can serve as a continuous alternative of discrete entropy estimation to control compression rate. And binarizer is adopted to quantize the output of encoder due to the binarization scheme is also directly defined by the importance map. Furthermore, a proxy function is introduced for binary operation in backward propagation to make it differentiable. Therefore, the encoder, decoder, binarizer and importance map can be jointly optimized in an end-to-end manner by using a subset of the ImageNet database. In low bit rate image compression, experiments show that our system significantly outperforms JPEG and JPEG 2000 by structural similarity (SSIM) index, and can produce the much better visual result with sharp edges, rich textures, and fewer artifacts.