Time-uniform Chernoff bounds via nonnegative supermartingales

TL;DR

This paper introduces a unified framework of exponential bounds for martingales crossing time-dependent thresholds, strengthening and generalizing many classical and modern tail inequalities across various settings.

Contribution

It presents a single assumption and theorem that unify and improve upon a wide range of existing martingale tail bounds, including scalar, matrix, and Banach-space cases.

Findings

Provides the strongest, most general time-uniform martingale inequalities to date.

Unifies classical, contemporary, and modern tail bounds under a single framework.

Quantifies concentration for scalar, matrix, and Banach-space martingales.

Abstract

We develop a class of exponential bounds for the probability that a martingale sequence crosses a time-dependent linear threshold. Our key insight is that it is both natural and fruitful to formulate exponential concentration inequalities in this way. We illustrate this point by presenting a single assumption and theorem that together unify and strengthen many tail bounds for martingales, including classical inequalities (1960-80) by Bernstein, Bennett, Hoeffding, and Freedman; contemporary inequalities (1980-2000) by Shorack and Wellner, Pinelis, Blackwell, van de Geer, and de la Pe\~na; and several modern inequalities (post-2000) by Khan, Tropp, Bercu and Touati, Delyon, and others. In each of these cases, we give the strongest and most general statements to date, quantifying the time-uniform concentration of scalar, matrix, and Banach-space-valued martingales, under a variety of nonparametric assumptions in discrete and continuous time. In doing so, we bridge the gap between existing line-crossing inequalities, the sequential probability ratio test, the Cram\'er-Chernoff method, self-normalized processes, and other parts of the literature.