A Note on the PAC Bayesian Theorem

TL;DR

This paper improves the PAC Bayesian Theorem by deriving tighter exponential moment inequalities for iid variables, reducing the dependence on sample size in the bound.

Contribution

It introduces new exponential moment inequalities and uses them to significantly tighten the PAC Bayesian bound.

Findings

Halves the logarithmic dependence on sample size in the bound.

Provides a more precise PAC Bayesian inequality.

Enhances theoretical understanding of PAC bounds.

Abstract

We prove general exponential moment inequalities for averages of [0,1]-valued iid random variables and use them to tighten the PAC Bayesian Theorem. The logarithmic dependence on the sample count in the enumerator of the PAC Bayesian bound is halved.