PAC-Bayes Iterated Logarithm Bounds for Martingale Mixtures

TL;DR

This paper introduces tight, PAC-Bayes-style concentration bounds for mixtures of martingales that are valid uniformly over all times and mixture distributions, extending Bernstein inequalities.

Contribution

It provides novel, simplified bounds for martingale mixtures that are uniform over time and distributions, improving upon prior results.

Findings

Bounds are tight and uniform over all finite times.

Extensions of Bernstein inequalities to martingale mixtures.

Simplifies previous concentration results.

Abstract

We give tight concentration bounds for mixtures of martingales that are simultaneously uniform over (a) mixture distributions, in a PAC-Bayes sense; and (b) all finite times. These bounds are proved in terms of the martingale variance, extending classical Bernstein inequalities, and sharpening and simplifying prior work.