A Distributed Computationally Aware Quantizer Design via Hyper Binning

TL;DR

This paper introduces hyper binning, a distributed quantization method that leverages correlation and function structure to optimize compression for distributed sources, improving functional approximation.

Contribution

It proposes hyper binning, a novel distributed quantizer that generalizes Slepian-Wolf encoding using linear discriminant analysis for better function-aware compression.

Findings

Hyper binning effectively captures source correlation and function structure.

The scheme reduces dimensionality while maintaining approximation accuracy.

Performance varies with source distribution, requiring tailored partitioning.

Abstract

We design a distributed function-aware quantization scheme for distributed functional compression. We consider $2$ correlated sources $X_1$ and $X_2$ and a destination that seeks an estimate $\hat{f}$ for the outcome of a continuous function $f(X_1,\,X_2)$. We develop a compression scheme called hyper binning in order to quantize $f$ via minimizing the entropy of joint source partitioning. Hyper binning is a natural generalization of Cover's random code construction for the asymptotically optimal Slepian-Wolf encoding scheme that makes use of orthogonal binning. The key idea behind this approach is to use linear discriminant analysis in order to characterize different source feature combinations. This scheme captures the correlation between the sources and the function's structure as a means of dimensionality reduction. We investigate the performance of hyper binning for different source distributions and identify which classes of sources entail more partitioning to achieve better function approximation. Our approach brings an information theory perspective to the traditional vector quantization technique from signal processing.