Algorithms for the Communication of Samples

TL;DR

This paper introduces two novel coding schemes for efficient communication of noisy data, improving upon existing methods and connecting importance sampling with Poisson functional representation.

Contribution

It presents ordered random coding and a hybrid scheme with dithered quantization, offering practical advantages and new insights in sample communication.

Findings

Ordered random coding reduces coding costs.

Hybrid scheme improves efficiency for bounded distributions.

Connects importance sampling with Poisson representation.

Abstract

The efficient communication of noisy data has applications in several areas of machine learning, such as neural compression or differential privacy, and is also known as reverse channel coding or the channel simulation problem. Here we propose two new coding schemes with practical advantages over existing approaches. First, we introduce ordered random coding (ORC) which uses a simple trick to reduce the coding cost of previous approaches. This scheme further illuminates a connection between schemes based on importance sampling and the so-called Poisson functional representation. Second, we describe a hybrid coding scheme which uses dithered quantization to more efficiently communicate samples from distributions with bounded support.