PAC-Bayesian Inequalities for Martingales

TL;DR

This paper develops new high-probability inequalities for martingales, extending PAC-Bayesian analysis beyond i.i.d. data to dependent, evolving processes, with applications in reinforcement learning and probability theory.

Contribution

It introduces PAC-Bayesian inequalities for martingales and a comparison inequality that tightens Hoeffding-Azuma bounds, broadening theoretical tools for dependent data analysis.

Findings

Derived high-probability inequalities for weighted averages of martingales.

Extended PAC-Bayesian analysis to non-i.i.d. settings.

Provided a tighter Hoeffding-Azuma inequality.

Abstract

We present a set of high-probability inequalities that control the concentration of weighted averages of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. Our results extend the PAC-Bayesian analysis in learning theory from the i.i.d. setting to martingales opening the way for its application to importance weighted sampling, reinforcement learning, and other interactive learning domains, as well as many other domains in probability theory and statistics, where martingales are encountered. We also present a comparison inequality that bounds the expectation of a convex function of a martingale difference sequence shifted to the [0,1] interval by the expectation of the same function of independent Bernoulli variables. This inequality is applied to derive a tighter analog of Hoeffding-Azuma's inequality.