Data Compression with Stochastic Codes

TL;DR

This paper provides a comprehensive overview of relative entropy coding, a promising new approach in data compression that leverages stochastic codes as an alternative to traditional quantisation and entropy coding methods.

Contribution

It offers an accessible introduction, practical insights, and a clear explanation of the algorithms behind relative entropy coding, highlighting its simplicity and potential in lossy data compression.

Findings

Highlights the computational and practical aspects of relative entropy coding.

Showcases contemporary applications in lossy data compression.

Emphasizes the simplicity and elegance of the underlying algorithms.

Abstract

Machine learning has had a major impact on data compression over the last decade and inspired many new, exciting theoretical and applied questions. This paper describes one such direction -- relative entropy coding -- which focuses on constructing stochastic codes, primarily as an alternative to quantisation and entropy coding in lossy source coding. Our primary aim is to provide a broad overview of the topic, with an emphasis on the computational and practical aspects currently missing from the literature. Our goal is threefold: for the curious reader, we aim to provide an intuitive picture of the field and convince them that relative entropy coding is a simple yet exciting emerging field in data compression research. For a reader interested in applied research on lossy data compression, we provide an account of the most salient contemporary applications. Finally, for the reader who has heard of relative entropy coding but has never been quite sure what it is or how the algorithms fit together, we hope to illustrate how simple and elegant the underlying constructions are.