The Capacity of Finite-State Channels in the High-Noise Regime

TL;DR

This paper derives a practical formula for the entropy rate derivative of hidden Markov processes, facilitating capacity analysis of finite-state channels, especially in high-noise conditions, with applications to binary and Gaussian noise channels.

Contribution

It provides a compact, easily evaluable formula for the entropy rate derivative, enabling capacity series expansion of finite-state channels in high-noise regimes.

Findings

Closed-form expression for the derivative in high-noise regime

Series expansion of channel capacity demonstrated

Application to binary-symmetric and Gaussian noise channels

Abstract

This paper considers the derivative of the entropy rate of a hidden Markov process with respect to the observation probabilities. The main result is a compact formula for the derivative that can be evaluated easily using Monte Carlo methods. It is applied to the problem of computing the capacity of a finite-state channel (FSC) and, in the high-noise regime, the formula has a simple closed-form expression that enables series expansion of the capacity of a FSC. This expansion is evaluated for a binary-symmetric channel under a (0,1) run-length limited constraint and an intersymbol-interference channel with Gaussian noise.