An efficient algorithm for the entropy rate of a hidden Markov model with unambiguous symbols

TL;DR

This paper introduces an efficient algorithm to compute the entropy rate of hidden Markov models with unambiguous symbols, enabling better estimation and capacity bounds for related communication channels.

Contribution

The paper presents a novel $O(Nq^3)$ algorithm for calculating the entropy rate of hidden Markov models with unambiguous symbols, improving computational efficiency.

Findings

Efficient formula for entropy rate calculation

Algorithm applicable to models with unknown parameters

Bounds on Gilbert channel capacity for q=2

Abstract

We demonstrate an efficient formula to compute the entropy rate $H(\mu)$ of a hidden Markov process with $q$ output symbols where at least one symbol is unambiguously received. Using an approximation to $H(\mu)$ to the first $N$ terms we give a $O(Nq^3$) algorithm to compute the entropy rate of the hidden Markov model. We use the algorithm to estimate the entropy rate when the parameters of the hidden Markov model are unknown.In the case of $q =2$ the process is the output of the Z-channel and we use this fact to give bounds on the capacity of the Gilbert channel.