A Class of Nonbinary Symmetric Information Bottleneck Problems

TL;DR

This paper investigates two dual information bottleneck problems involving a Markov chain with a modulo-additive noise channel, providing analytical solutions for optimal information rates and distributions.

Contribution

It introduces a unified analysis of symmetric nonbinary information bottleneck and privacy funnel problems with explicit solutions for modulo-additive noise channels.

Findings

Analytical characterization of optimal information rates.

Explicit optimal distributions for the considered problems.

Insights into the duality between information bottleneck and privacy funnel.

Abstract

We study two dual settings of information processing. Let $ \mathsf{Y} \rightarrow \mathsf{X} \rightarrow \mathsf{W} $ be a Markov chain with fixed joint probability mass function $ \mathsf{P}_{\mathsf{X}\mathsf{Y}} $ and a mutual information constraint on the pair $ (\mathsf{W},\mathsf{X}) $. For the first problem, known as Information Bottleneck, we aim to maximize the mutual information between the random variables $ \mathsf{Y} $ and $ \mathsf{W} $, while for the second problem, termed as Privacy Funnel, our goal is to minimize it. In particular, we analyze the scenario for which $ \mathsf{X} $ is the input, and $ \mathsf{Y} $ is the output of modulo-additive noise channel. We provide analytical characterization of the optimal information rates and the achieving distributions.