Fundamental Limitations in Sequential Prediction and Recursive Algorithms: $\mathcal{L}_{p}$ Bounds via an Entropic Analysis

TL;DR

This paper establishes fundamental $ ext{L}_p$ bounds for sequential prediction and recursive algorithms using entropic analysis, providing insights into the limits of performance based on data and noise entropy.

Contribution

It introduces a novel entropic framework to derive lower bounds on $ ext{L}_p$ performance in sequential and recursive algorithms, linking bounds to conditional entropy.

Findings

Derived $ ext{L}_p$ bounds via entropic relationships.

Characterized conditions for achieving the bounds.

Provided a unified entropic analysis approach.

Abstract

In this paper, we obtain fundamental $\mathcal{L}_{p}$ bounds in sequential prediction and recursive algorithms via an entropic analysis. Both classes of problems are examined by investigating the underlying entropic relationships of the data and/or noises involved, and the derived lower bounds may all be quantified in a conditional entropy characterization. We also study the conditions to achieve the generic bounds from an innovations' viewpoint.