On the randomness of independent experiments

TL;DR

This paper investigates the limits of randomness and information storage for independent experiments by deriving bounds on smooth entropies of product distributions, which are key in understanding data compression and randomness extraction.

Contribution

It provides explicit, nearly tight bounds on smooth entropies for n-fold product distributions, advancing the theoretical understanding of information-theoretic limits.

Findings

Derived explicit bounds on smooth entropies

Bounds are nearly tight for product distributions

Enhances understanding of data compression and randomness extraction

Abstract

Given a probability distribution P, what is the minimum amount of bits needed to store a value x sampled according to P, such that x can later be recovered (except with some small probability)? Or, what is the maximum amount of uniform randomness that can be extracted from x? Answering these and similar information-theoretic questions typically boils down to computing so-called smooth entropies. In this paper, we derive explicit and almost tight bounds on the smooth entropies of n-fold product distributions.