On Information Bottleneck for Gaussian Processes

TL;DR

This paper explores the information bottleneck problem for Gaussian processes, providing solutions in both frequency and time domains, and addressing uncertainty in source distribution with an optimal white spectrum strategy.

Contribution

It introduces a water-filling solution for the IB rate in Gaussian sources and proposes a linear prediction-based time-domain realization, also addressing uncertain source distributions.

Findings

Water-filling solution for IB rate in Gaussian sources

Time-domain realization using linear prediction

White SNR spectrum optimal under distribution uncertainty

Abstract

The information bottleneck problem (IB) of jointly stationary Gaussian sources is considered. A water-filling solution for the IB rate is given in terms of its SNR spectrum and whose rate is attained via frequency domain test-channel realization. A time-domain realization of the IB rate, based on linear prediction, is also proposed, which lends itself to an efficient implementation of the corresponding remote source-coding problem. A compound version of the problem is addressed, in which the joint distribution of the source is not precisely specified but rather in terms of a lower bound on the guaranteed mutual information. It is proved that a white SNR spectrum is optimal for this setting.