Concentration and Confidence for Discrete Bayesian Sequence Predictors

TL;DR

This paper establishes tight high-probability bounds on the cumulative KL divergence error in Bayesian sequence prediction and develops confidence bounds, enhancing its applicability in the KWIK learning framework.

Contribution

It provides the first tight high-probability bounds on cumulative KL divergence and constructs confidence bounds for Bayesian sequence predictors, advancing theoretical understanding.

Findings

Proved tight high-probability bounds on cumulative KL divergence error.

Constructed confidence bounds for KL and Hellinger errors.

Applied results to improve Bayesian prediction in the KWIK framework.

Abstract

Bayesian sequence prediction is a simple technique for predicting future symbols sampled from an unknown measure on infinite sequences over a countable alphabet. While strong bounds on the expected cumulative error are known, there are only limited results on the distribution of this error. We prove tight high-probability bounds on the cumulative error, which is measured in terms of the Kullback-Leibler (KL) divergence. We also consider the problem of constructing upper confidence bounds on the KL and Hellinger errors similar to those constructed from Hoeffding-like bounds in the i.i.d. case. The new results are applied to show that Bayesian sequence prediction can be used in the Knows What It Knows (KWIK) framework with bounds that match the state-of-the-art.