Low and Upper Bound of Approximate Sequence for the Entropy Rate of Binary Hidden Markov Processes

TL;DR

This paper establishes bounds on the entropy rate of binary hidden Markov processes, demonstrating rapid convergence and providing a theoretical foundation for accurate entropy estimation.

Contribution

It introduces low and upper bound sequences for the entropy of binary hidden Markov models, enhancing understanding of convergence and estimation accuracy.

Findings

Bound sequences converge quickly due to geometric decay.

Error bias decreases exponentially, improving entropy estimates.

Provides a theoretical basis for entropy rate approximation.

Abstract

In the paper, the approximate sequence for entropy of some binary hidden Markov models has been found to have two bound sequences, the low bound sequence and the upper bound sequence. The error bias of the approximate sequence is bound by a geometric sequence with a scale factor less than 1 which decreases quickly to zero. It helps to understand the convergence of entropy rate of generic hidden Markov models, and it provides a theoretical base for estimating the entropy rate of some hidden Markov models at any accuracy.