On the Joint Entropy of $d$-Wise-Independent Variables

TL;DR

This paper investigates the minimum joint entropy of $d$-wise independent discrete variables under distribution constraints, improving bounds and establishing tight bounds in specific cases, especially for binary variables.

Contribution

It advances the understanding of joint entropy bounds for $d$-wise independent variables, providing new bounds, extending previous results, and identifying cases with tight bounds.

Findings

Improved bounds on joint entropy for $d$-wise independent variables.

Established tight lower bounds for binary variables with pairwise and three-wise independence.

Extended results to a wider parameter range and identified cases with matching upper bounds.

Abstract

How low can the joint entropy of $n$ $d$-wise independent (for $d\ge2$) discrete random variables be, subject to given constraints on the individual distributions (say, no value may be taken by a variable with probability greater than $p$, for $p<1$)? This question has been posed and partially answered in a recent work of Babai. In this paper we improve some of his bounds, prove new bounds in a wider range of parameters and show matching upper bounds in some special cases. In particular, we prove tight lower bounds for the min-entropy (as well as the entropy) of pairwise and three-wise independent balanced binary variables for infinitely many values of $n$.