A Note on the Chernoff Bound for Random Variables in the Unit Interval

TL;DR

This paper extends the Chernoff bound to random variables in the unit interval, providing a proof to facilitate its use in statistical learning theory beyond Bernoulli variables.

Contribution

It offers a proof of the Chernoff bound extension for variables in [0,1], which was previously less well known in the community.

Findings

Provides a proof of the Chernoff bound extension for [0,1] variables.

Facilitates the application of Chernoff bounds in broader statistical learning contexts.

Enhances understanding of probabilistic bounds for non-binary random variables.

Abstract

The Chernoff bound is a well-known tool for obtaining a high probability bound on the expectation of a Bernoulli random variable in terms of its sample average. This bound is commonly used in statistical learning theory to upper bound the generalisation risk of a hypothesis in terms of its empirical risk on held-out data, for the case of a binary-valued loss function. However, the extension of this bound to the case of random variables taking values in the unit interval is less well known in the community. In this note we provide a proof of this extension for convenience and future reference.