On Sequential Estimation and Prediction for Discrete Time Series

TL;DR

This paper surveys recent research on sequential estimation and prediction for stationary discrete time series, highlighting the limitations and possibilities of universal estimators under various conditions.

Contribution

It provides a comprehensive overview of recent advances in universal estimation for stationary processes, including methods for restricted classes and along stopping times.

Findings

Universal estimators cannot converge almost surely for all stationary processes.

Restricted classes of processes allow for universal estimation.

Estimation along stopping times offers potential for universal methods.

Abstract

The problem of extracting as much information as possible from a sequence of observations of a stationary stochastic process $X_0,X_1,...X_n$ has been considered by many authors from different points of view. It has long been known through the work of D. Bailey that no universal estimator for $\textbf{P}(X_{n+1}|X_0,X_1,...X_n)$ can be found which converges to the true estimator almost surely. Despite this result, for restricted classes of processes, or for sequences of estimators along stopping times, universal estimators can be found. We present here a survey of some of the recent work that has been done along these lines.