Variational Bayesian Quantization

TL;DR

This paper introduces a flexible variational Bayesian quantization algorithm for deep probabilistic models like VAEs, enabling efficient data and model compression with adjustable rate-distortion trade-offs, outperforming traditional methods.

Contribution

The paper presents a novel quantization method that separates model training from quantization, allowing plug-and-play compression and extending arithmetic coding to continuous domains.

Findings

Outperforms JPEG in image compression across various bit rates

Effectively utilizes posterior uncertainty for improved quantization

Demonstrates versatility on Bayesian neural word embeddings

Abstract

We propose a novel algorithm for quantizing continuous latent representations in trained models. Our approach applies to deep probabilistic models, such as variational autoencoders (VAEs), and enables both data and model compression. Unlike current end-to-end neural compression methods that cater the model to a fixed quantization scheme, our algorithm separates model design and training from quantization. Consequently, our algorithm enables "plug-and-play" compression with variable rate-distortion trade-off, using a single trained model. Our algorithm can be seen as a novel extension of arithmetic coding to the continuous domain, and uses adaptive quantization accuracy based on estimates of posterior uncertainty. Our experimental results demonstrate the importance of taking into account posterior uncertainties, and show that image compression with the proposed algorithm outperforms JPEG over a wide range of bit rates using only a single standard VAE. Further experiments on Bayesian neural word embeddings demonstrate the versatility of the proposed method.