A proof of the Shepp-Olkin entropy monotonicity conjecture

TL;DR

This paper proves the Shepp-Olkin conjecture that the Shannon entropy of a sum of biased coins increases as the coins become fairer, confirming the intuition about randomness and entropy.

Contribution

The authors provide a rigorous proof of the Shepp-Olkin entropy monotonicity conjecture, using a novel construction related to previous work on entropy concavity.

Findings

Shannon entropy increases with fairness of coins.

The proof confirms the conjecture for the sum of Bernoulli variables.

Discussion on potential generalizations to q-Rényi and q-Tsallis entropies.

Abstract

Consider tossing a collection of coins, each fair or biased towards heads, and take the distribution of the total number of heads that result. It is natural to conjecture that this distribution should be 'more random' when each coin is fairer. Indeed, Shepp and Olkin conjectured that the Shannon entropy of this distribution is monotonically increasing in this case. We resolve this conjecture, by proving that this intuition is correct. Our proof uses a construction which was previously developed by the authors to prove a related conjecture of Shepp and Olkin concerning concavity of entropy. We discuss whether this result can be generalized to $q$-R\'{e}nyi and $q$-Tsallis entropies, for a range of values of $q$.