Information Bottleneck on General Alphabets
Georg Pichler, G\"unther Koliander

TL;DR
This paper rigorously extends a source coding theorem related to the Information Bottleneck method to arbitrary alphabets, establishing conditions under which certain information constraints are achievable.
Contribution
It generalizes a known discrete source coding theorem to arbitrary alphabets and introduces a technique for lifting discrete results to continuous or general spaces.
Findings
Proves a source coding theorem for general alphabets.
Establishes the existence of a function with specified rate and mutual information constraints.
Provides a method to extend discrete coding theorems to arbitrary alphabets.
Abstract
We prove rigorously a source coding theorem that can probably be considered folklore, a generalization to arbitrary alphabets of a problem motivated by the Information Bottleneck method. For general random variables , we show essentially that for some , a function with rate limit and exists if and only if there is a random variable such that the Markov chain holds, and . The proof relies on the well established discrete case and showcases a technique for lifting discrete coding theorems to arbitrary alphabets.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
