First-Order Theory of Probabilistic Independence and Single-Letter Characterizations of Capacity Regions

TL;DR

This paper explores the expressive power of the first-order theory of probabilistic independence and introduces a single-letter characterization of capacity regions for various multiuser channels, along with a complexity hierarchy.

Contribution

It demonstrates the undecidability of the first-order theory of probabilistic independence and provides a novel single-letter characterization of capacity regions using first-order logic.

Findings

The first-order theory of probabilistic independence is highly expressive and undecidable.

A single-letter characterization of capacity regions is constructed for multiple multiuser channels.

A linear entropy hierarchy classifies the complexity of single-letter characterizations.

Abstract

We consider the first-order theory of random variables with the probabilistic independence relation, which concerns statements consisting of random variables, the probabilistic independence symbol, logical operators, and existential and universal quantifiers. Although probabilistic independence is the only non-logical relation included, this theory is surprisingly expressive, and is able to interpret the true first-order arithmetic over natural numbers (and hence is undecidable). We also construct a single-letter characterization of the capacity region for a general class of multiuser coding settings (including broadcast channel, interference channel and relay channel), using a first-order formula. We then introduce the linear entropy hierarchy to classify single-letter characterizations according to their complexity.