Entropies of weighted sums in cyclic groups and an application to polar codes

TL;DR

This paper investigates the relationship between the Shannon entropies of sums and differences of i.i.d. random variables in cyclic groups and applies these findings to improve polar code constructions with better error performance.

Contribution

It establishes new entropy relations in cyclic groups and introduces non-canonical kernels for polar codes, enhancing their error probability performance.

Findings

Entropy of sum and difference can differ arbitrarily in integer groups.

In Z/3Z, the difference entropy always exceeds or equals the sum entropy.

Using non-canonical kernels improves polar code error probabilities.

Abstract

In this note, the following basic question is explored: in a cyclic group, how are the Shannon entropies of the sum and difference of i.i.d. random variables related to each other? For the integer group, we show that they can differ by any real number additively, but not too much multiplicatively; on the other hand, for $\mathbb{Z}/3\mathbb{Z}$, the entropy of the difference is always at least as large as that of the sum. These results are closely related to the study of more-sum-than-difference (i.e. MSTD) sets in additive combinatorics. We also investigate polar codes for $q$-ary input channels using non-canonical kernels to construct the generator matrix, and present applications of our results to constructing polar codes with significantly improved error probability compared to the canonical construction.