From finite-system entropy to entropy rate for a Hidden Markov Process

TL;DR

This paper proves a conjecture relating the entropy rate of a Hidden Markov Process to finite-length process entropy, enabling power series expansion in noise and advancing theoretical understanding.

Contribution

It generalizes and proves a conjecture connecting the entropy rate of HMPs to finite-length process entropy, facilitating analytical expansions.

Findings

Validated the conjecture relating HMP entropy rate to finite-length process entropy

Derived power series expansion for HMP entropy rate in noise variable

Discussed implications for theoretical analysis and practical computation

Abstract

A recent result presented the expansion for the entropy rate of a Hidden Markov Process (HMP) as a power series in the noise variable $\eps$. The coefficients of the expansion around the noiseless ($\eps = 0$) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of a HMP to the entropy of a process of finite length (which is calculated analytically). In this communication we generalize and prove the validity of the conjecture, and discuss the theoretical and practical consequences of our new theorem.