Conditional Entropies of k-Deletion/Insertion Channels

TL;DR

This paper investigates the entropy of sequences transmitted over k-deletion and k-insertion channels, revealing how sequence structure affects entropy and confirming a conjecture about alternating sequences maximizing input entropy in the 1-deletion channel.

Contribution

It proves that for 1-deletion and 1-insertion channels, entropy is minimized for skewed run-length sequences and maximized for balanced ones, confirming a conjecture for the 1-deletion channel.

Findings

Input entropy minimized for skewed run-length sequences.

Input entropy maximized for balanced run-length sequences.

Confirmed the conjecture that alternating sequences maximize input entropy in the 1-deletion channel.

Abstract

The channel output entropy of a transmitted sequence is the entropy of the possible channel outputs and similarly the channel input entropy of a received sequence is the entropy of all possible transmitted sequences. The goal of this work is to study these entropy values for the k-deletion, k-insertion channels, where exactly k symbols are deleted, inserted in the transmitted sequence, respectively. If all possible sequences are transmitted with the same probability then studying the input and output entropies is equivalent. For both the 1-deletion and 1-insertion channels, it is proved that among all sequences with a fixed number of runs, the input entropy is minimized for sequences with a skewed distribution of their run lengths and it is maximized for sequences with a balanced distribution of their run lengths. Among our results, we establish a conjecture by Atashpendar et al. which claims that for the 1-deletion channel, the input entropy is maximized by the alternating sequences over all binary sequences. This conjecture is also verified for the 2-deletion channel, where it is proved that constant sequences with a single run minimize the input entropy.