Generic Bounds on the Maximum Deviations in Sequential Prediction: An Information-Theoretic Analysis

TL;DR

This paper establishes fundamental information-theoretic bounds on maximum prediction deviations in sequential prediction, showing these bounds depend solely on conditional entropy and are achieved under specific error conditions.

Contribution

It introduces generic bounds on prediction errors based on conditional entropy, providing a new theoretical framework for understanding deviations in sequential prediction.

Findings

Bounds depend only on conditional entropy

Asymptotic bounds are achieved with white, uniform errors

Provides a theoretical foundation for prediction error analysis

Abstract

In this paper, we derive generic bounds on the maximum deviations in prediction errors for sequential prediction via an information-theoretic approach. The fundamental bounds are shown to depend only on the conditional entropy of the data point to be predicted given the previous data points. In the asymptotic case, the bounds are achieved if and only if the prediction error is white and uniformly distributed.