Non-existence of certain kind of finite-letter mutual information characterization for a class of time-invariant Markoff channels

TL;DR

This paper investigates the limitations of finite-letter mutual information characterizations for certain Markoff channels, proving that some capacity characterizations cannot be represented for specific finite state machine channels due to undecidability.

Contribution

It introduces a formal definition of a class of capacity characterizations based on conditional mutual information and demonstrates their non-existence for some FSMC families.

Findings

Approximation of these characterizations is computable for point-to-point channels.

Certain FSMC capacities cannot be captured by finite-letter mutual information characterizations.

Undecidability results imply fundamental limits on characterizing capacities of some channels.

Abstract

We provide a rigorous definition of a certain kind of characterization for capacity regions of a family of Markoff networks which is based on optimization problems resulting out of calculating conditional mutual information from a finite number of random variables and by constraining these random variables in a certain way. This definition is partly motivated by the definition of single-letter characterizations in information theory. For a point-to-point Markoff channel, we prove that approximating the solution to these characterizations within an additive constant is a computable problem. Based on previous undecidability results concerning capacities of certain class of finite state machine channels (FSMCs), it will follow that there exists an example of family of FSMCs for which given such a characterization, this characterization cannot represent the capacity of this family of FSMCs.