Exponential Bounds for Convergence of Entropy Rate Approximations in Hidden Markov Models Satisfying a Path-Mergeability Condition

TL;DR

This paper proves that for a specific class of hidden Markov models with path-mergeable states, the entropy rate estimates converge exponentially fast, and this property is common and easy to verify.

Contribution

It establishes exponential convergence bounds for entropy rate approximations in path-mergeable HMMs and shows this property is typical and testable in HMM topology space.

Findings

Exponential convergence of entropy rate estimates in path-mergeable HMMs

Path-mergeability is typical in HMM topologies

Path-mergeability is easily testable

Abstract

A hidden Markov model (HMM) is said to have path-mergeable states if for any two states i,j there exists a word w and state k such that it is possible to transition from both i and j to k while emitting w. We show that for a finite HMM with path-mergeable states the block estimates of the entropy rate converge exponentially fast. We also show that the path-mergeability property is asymptotically typical in the space of HMM topolgies and easily testable.