Maximum Entropy of Sums of Independent Ternary Random Variables

TL;DR

This paper solves the problem of maximizing the Shannon entropy of sums of independent ternary random variables, extending known results to a three-symbol alphabet with a precise characterization of the maximizing distribution.

Contribution

It establishes the maximum entropy distribution for sums of independent ternary variables, extending the Shepp–Olkin–Mateev theorem to this case.

Findings

Maximum entropy is achieved with specific Bernoulli distributions.

The entropy of the sum is maximized when certain variables are uniform on {0,2}.

The result generalizes previous binary cases to ternary alphabets.

Abstract

The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if \(X_1,\ldots,X_n\) are independent random variables taking values in \(\{0,1,2\}\), then the entropy of \(S_n=X_1+\cdots+X_n\) is maximized when \(X_1,\ldots,X_{n-1}\) are uniform on \(\{0,2\}\) and the probability mass function of \(X_n\) is given by \(\Prob(X_n=0) = \Prob(X_n=2) = w/2\), \(\Prob(X_n=1) = 1-w\), where \(w = \big(1 + 2^{-H(B_n)+H(B_{n-1})}\big)^{-1}\) and \(B_m\sim \Bin(m,1/2)\). The statement can be seen as an extension to ternary alphabets of the Shepp--Olkin--Mateev theorem. The proof uses the Hermite--Biehler theorem, Newton's inequalities, and Yu's maximum-entropy theorem for ultra-log-concave distributions.