The Porosity of Additive Noise Sequences

TL;DR

This paper introduces the concept of porosity to measure the capacity of additive noise channels with individual noise sequences, extending universal coding principles to deterministic noise and providing schemes that achieve this capacity.

Contribution

It defines porosity as a new measure for deterministic noise sequences and develops universal coding schemes that attain this capacity, extending classical information theory results.

Findings

Porosity quantifies the capacity of individual noise sequences.

Universal schemes can achieve porosity for any noise sequence.

Connections are made to finite-state predictability and universal source coding.

Abstract

Consider a binary additive noise channel with noiseless feedback. When the noise is a stationary and ergodic process $\mathbf{Z}$, the capacity is $1-\mathbb{H}(\mathbf{Z})$ ($\mathbb{H}(\cdot)$ denoting the entropy rate). It is shown analogously that when the noise is a deterministic sequence $z^\infty$, the capacity under finite-state encoding and decoding is $1-\bar{\rho}(z^\infty)$, where $\bar{\rho}(\cdot)$ is Lempel and Ziv's finite-state compressibility. This quantity is termed the \emph{porosity} $\underline{\sigma}(\cdot)$ of an individual noise sequence. A sequence of schemes are presented that universally achieve porosity for any noise sequence. These converse and achievability results may be interpreted both as a channel-coding counterpart to Ziv and Lempel's work in universal source coding, as well as an extension to the work by Lomnitz and Feder and Shayevitz and Feder on communication across modulo-additive channels. Additionally, a slightly more practical architecture is suggested that draws a connection with finite-state predictability, as introduced by Feder, Gutman, and Merhav.