The Input and Output Entropies of the $k$-Deletion/Insertion Channel

TL;DR

This paper investigates the entropy of inputs and outputs in k-deletion/insertion channels, revealing how run-length distributions affect entropy and confirming conjectures about entropy maximization in binary channels.

Contribution

It proves that for 1-insertion and 1-deletion channels, entropy is minimized for skewed run-length words and maximized for balanced ones, confirming a conjecture for binary 1-deletion channels.

Findings

Input entropy minimized for skewed run-length words.

Input entropy maximized for balanced run-length words.

Confirmed conjecture that alternating words maximize entropy in binary 1-deletion channels.

Abstract

The channel output entropy of a transmitted word is the entropy of the possible channel outputs and similarly, the input entropy of a received word is the entropy of all possible transmitted words. The goal of this work is to study these entropy values for the k-deletion, k-insertion channel, where exactly k symbols are deleted, and inserted in the transmitted word, respectively. If all possible words are transmitted with the same probability then studying the input and output entropies is equivalent. For both the 1-insertion and 1-deletion channels, it is proved that among all words with a fixed number of runs, the input entropy is minimized for words with a skewed distribution of their run lengths and it is maximized for words with a balanced distribution of their run lengths. Among our results, we establish a conjecture by Atashpendar et al. which claims that for the binary 1-deletion, the input entropy is maximized for the alternating words. This conjecture is also verified for the 2-deletion channel, where it is proved that constant words with a single run minimize the input entropy.