Capacity Approximations for Insertion Channels with Small Insertion Probabilities

TL;DR

This paper derives the capacity of binary insertion channels with small insertion probabilities, extending previous deletion channel results, and compares two models to quantify their capacity differences in the asymptotic regime.

Contribution

It provides the first asymptotic capacity expansion for binary insertion channels with small insertion probabilities, extending methods from deletion channel analysis.

Findings

Capacity of random insertion channel is higher than Gallager insertion channel.

Capacity differs only in higher order terms from achievable rates with i.i.d. inputs.

Quantifies the capacity difference between two insertion models in the asymptotic regime.

Abstract

Channels with synchronization errors, exhibiting deletion and insertion errors, find practical applications in DNA storage, data reconstruction, and various other domains. Presence of insertions and deletions render the channel with memory, complicating capacity analysis. For instance, despite the formulation of an independent and identically distributed (i.i.d.) deletion channel more than fifty years ago, and proof that the channel is information stable, hence its Shannon capacity exists, calculation of the capacity remained elusive. However, a relatively recent result establishes the capacity of the deletion channel in the asymptotic regime of small deletion probabilities by computing the dominant terms of the capacity expansion. This paper extends that result to binary insertion channels, determining the dominant terms of the channel capacity for small insertion probabilities and establishing capacity in this asymptotic regime. Specifically, we consider two i.i.d. insertion channel models: insertion channel with possible random bit insertions after every transmitted bit and the Gallager insertion model, for which a bit is replaced by two random bits with a certain probability. To prove our results, we build on methods used for the deletion channel, employing Bernoulli(1/2) inputs for achievability and coupling this with a converse using stationary and ergodic processes as inputs, and show that the channel capacity differs only in the higher order terms from the achievable rates with i.i.d. inputs. The results, for instance, show that the capacity of the random insertion channel is higher than that of the Gallager insertion channel, and quantifies the difference in the asymptotic regime.