A note on a confidence bound of Kuzborskij and Szepesv\'ari

TL;DR

This paper discusses and simplifies the presentation of a recent confidence bound for functions of independent, potentially unbounded, and non-identically distributed random variables, based on concentration inequalities and PAC-Bayes methods.

Contribution

It provides streamlined proofs and an exposition of Kuzborskij and Szepesvári's novel confidence bound that applies to unbounded, non-i.i.d. random variables.

Findings

Confidence bound applicable to unbounded, non-i.i.d. variables

PAC-Bayes-ification of the confidence bound

Simplified proofs and exposition

Abstract

In an interesting recent work, Kuzborskij and Szepesv\'ari derived a confidence bound for functions of independent random variables, which is based on an inequality that relates concentration to squared perturbations of the chosen function. Kuzborskij and Szepesv\'ari also established the PAC-Bayes-ification of their confidence bound. Two important aspects of their work are that the random variables could be of unbounded range, and not necessarily of an identical distribution. The purpose of this note is to advertise/discuss these interesting results, with streamlined proofs. This expository note is written for persons who, metaphorically speaking, enjoy the "featured movie" but prefer to skip the preview sequence.