Taylor series expansions for the entropy rate of Hidden Markov Processes

TL;DR

This paper develops series expansions for the entropy rate of Hidden Markov Processes in different regimes, generalizing previous conjectures and analyzing convergence properties, advancing theoretical understanding of these processes.

Contribution

It generalizes and proves a conjecture linking entropy rate to finite system entropies, and establishes new series expansions with convergence analysis.

Findings

Series expansions for entropy rate in two regimes

Proof of a conjecture relating entropy rate to finite system entropies

Analysis of convergence radius of the series expansions

Abstract

Finding the entropy rate of Hidden Markov Processes is an active research topic, of both theoretical and practical importance. A recently used approach is studying the asymptotic behavior of the entropy rate in various regimes. In this paper we generalize and prove a previous conjecture relating the entropy rate to entropies of finite systems. Building on our new theorems, we establish series expansions for the entropy rate in two different regimes. We also study the radius of convergence of the two series expansions.