On the Bernstein-Hoeffding method

TL;DR

This paper extends the Bernstein-Hoeffding method to broader classes of moments, improving Hoeffding's inequality and connecting it with Bernstein polynomials, higher moments, and conditional distributions for sharper probabilistic bounds.

Contribution

It generalizes and enhances Hoeffding's inequality using the Bernstein-Hoeffding method, incorporating higher moments and conditional information for tighter bounds.

Findings

Hoeffding's bound is shown to be optimal in a broader context.

The method can incorporate higher moments of the variables.

Under certain conditions, the method reduces to Markov's inequality.

Abstract

We show that the Bernstein-Hoeffding method can be employed to a larger class of generalized moments. This class includes the exponential moments whose properties play a key role in the proof of a well-known inequality of Wassily Hoeffding, for sums of independent and bounded random variables whose mean is assumed to be known. As a result we can generalise and improve upon this inequality. We show that Hoeffding's bound is optimal in a broader sense. Our approach allows to obtain "missing" factors in Hoeffding's inequality whose existence is motivated by the central limit theorem. The later result is a rather weaker version of a theorem that is due to Michel Talagrand. Using ideas from the theory of Bernstein polynomials, we show that the Bernstein-Hoeffding method can be adapted to case in which one has information on higher moments of the random variables. Moreover, we consider the performance of the method under additional information on the conditional distribution of the random variables and, finally, we show that the method reduces to Markov's inequality when employed to non-negative and unbounded random variables.