The Entropy of a Binary Hidden Markov Process

TL;DR

This paper calculates the entropy of a binary symmetric Hidden Markov Process by mapping it to an Ising model, deriving an expansion in the noise parameter, and analyzing the series convergence.

Contribution

It introduces a novel mapping of the Hidden Markov Process to an Ising model and extends the entropy expansion to high orders using a conjecture.

Findings

Entropy expansion coefficients calculated up to second order

Extended to 11th order using a conjecture

Discussed convergence properties of the series

Abstract

The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter epsilon. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion coefficients up to second order in epsilon. Using a conjecture we extend the calculation to 11th order and discuss the convergence of the resulting series.