Compressing Images by Encoding Their Latent Representations with Relative Entropy Coding

TL;DR

This paper introduces Relative Entropy Coding (REC), a novel method for efficiently encoding latent representations of images, enabling effective single-image compression and competitive lossy compression performance.

Contribution

REC is a new encoding method that directly compresses latent representations close to their relative entropy, applicable to single-image lossy compression.

Findings

REC achieves near-theoretical compression efficiency on multiple datasets.

REC is effective for single-image compression, unlike previous bits-back methods.

REC performs competitively with state-of-the-art lossy compression methods on Kodak.

Abstract

Variational Autoencoders (VAEs) have seen widespread use in learned image compression. They are used to learn expressive latent representations on which downstream compression methods can operate with high efficiency. Recently proposed 'bits-back' methods can indirectly encode the latent representation of images with codelength close to the relative entropy between the latent posterior and the prior. However, due to the underlying algorithm, these methods can only be used for lossless compression, and they only achieve their nominal efficiency when compressing multiple images simultaneously; they are inefficient for compressing single images. As an alternative, we propose a novel method, Relative Entropy Coding (REC), that can directly encode the latent representation with codelength close to the relative entropy for single images, supported by our empirical results obtained on the Cifar10, ImageNet32 and Kodak datasets. Moreover, unlike previous bits-back methods, REC is immediately applicable to lossy compression, where it is competitive with the state-of-the-art on the Kodak dataset.