Upper Bounds on the Feedback Error Exponent of Channels With States and Memory

TL;DR

This paper derives upper bounds on the feedback error exponent and capacity for channels with unknown, evolving states, extending classical results to more general stochastic state processes using martingale techniques.

Contribution

It introduces upper bounds for feedback error exponents and capacity in channels with non-Markovian, non-ergodic states, generalizing Burnashev's bounds to broader stochastic processes.

Findings

Bounds expressed via directed mutual information and relative entropy

Bounds reduce to Burnashev's expression for memoryless channels

Uses martingale tools to analyze entropy-based stochastic processes

Abstract

As a class of state-dependent channels, Markov channels have been long studied in information theory for characterizing the feedback capacity and error exponent. This paper studies a more general variant of such channels where the state evolves via a general stochastic process, not necessarily Markov or ergodic. The states are assumed to be unknown to the transmitter and the receiver, but the underlying probability distributions are known. For this setup, we derive an upper bound on the feedback error exponent and the feedback capacity with variable-length codes. The bounds are expressed in terms of the directed mutual information and directed relative entropy. The bounds on the error exponent are simplified to Burnashev's expression for discrete memoryless channels. Our method relies on tools from the theory of martingales to analyze a stochastic process defined based on the entropy of the message given the past channel's outputs.