The Conditional Regret-Capacity Theorem for Batch Universal Prediction

TL;DR

This paper introduces a conditional regret-capacity theorem for universal prediction, providing lower bounds on batch regret, with applications to binary sources and a connection to Rényi information measures.

Contribution

It extends the classical regret-capacity theorem to a conditional setting and links it to Rényi divergence, advancing theoretical understanding in universal prediction.

Findings

Derived a conditional regret-capacity theorem

Applied the theorem to binary memoryless sources

Connected Rényi divergence with Sibson's mutual information

Abstract

We derive a conditional version of the classical regret-capacity theorem. This result can be used in universal prediction to find lower bounds on the minimal batch regret, which is a recently introduced generalization of the average regret, when batches of training data are available to the predictor. As an example, we apply this result to the class of binary memoryless sources. Finally, we generalize the theorem to R\'enyi information measures, revealing a deep connection between the conditional R\'enyi divergence and the conditional Sibson's mutual information.