Improved Hoeffding's Lemma and Hoeffding's Tail Bounds

TL;DR

This paper presents an improved version of Hoeffding's lemma and tail bounds specifically for left skewed zero mean random variables, leading to tighter probabilistic bounds for sums of such variables.

Contribution

The authors derive a novel improvement to Hoeffding's lemma using Taylor's expansion and convexity, resulting in sharper tail bounds for sums of independent zero mean variables with skewed distributions.

Findings

Improved tail bounds for left skewed zero mean variables

Enhanced two-sided bounds for non-symmetric variables

Tighter probabilistic estimates for sums of skewed random variables

Abstract

The purpose of this letter is to improve Hoeffding's lemma and consequently Hoeffding's tail bounds. The improvement pertains to left skewed zero mean random variables $X\in[a,b]$, where $a<0$ and $-a>b$. The proof of Hoeffding's improved lemma uses Taylor's expansion, the convexity of $\exp(sx), s\in {\bf R}$ and an unnoticed observation since Hoeffding's publication in 1963 that for $-a>b$ the maximum of the intermediate function $\tau(1-\tau)$ appearing in Hoeffding's proof is attained at an endpoint rather than at $\tau=0.5$ as in the case $b>-a$. Using Hoeffding's improved lemma we obtain one sided and two sided tail bounds for $P(S_n\ge t)$ and $P(|S_n|\ge t)$, respectively, where $S_n=\sum_{i=1}^nX_i$ and the $X_i\in[a_i,b_i],i=1,...,n$ are independent zero mean random variables (not necessarily identically distributed). It is interesting to note that we could also improve Hoeffding's two sided bound for all $\{X_i: a_i\ne b_i,i=1,...,n\}$. This is so because here the one sided bound should be increased by $P(-S_n\ge t)$, wherein the left skewed intervals become right skewed and vice versa.