Maximum Entropy Rate of Markov Sources for Systems With Non-regular Constraints

TL;DR

This paper extends the understanding of the maximum entropy rate of Markov sources to systems with non-regular constraints, demonstrating that the capacity limit applies beyond regular languages using generating functions and tree-based models.

Contribution

It generalizes Shannon's capacity results from regular to non-regular constrained systems through novel mathematical modeling.

Findings

Capacity limit applies to non-regular constraints

Generating functions effectively model constrained systems

Random walks on trees provide insights into system capacity

Abstract

Using the concept of discrete noiseless channels, it was shown by Shannon in A Mathematical Theory of Communication that the ultimate performance of an encoder for a constrained system is limited by the combinatorial capacity of the system if the constraints define a regular language. In the present work, it is shown that this is not an inherent property of regularity but holds in general. To show this, constrained systems are described by generating functions and random walks on trees.