A simple randomized algorithm for sequential prediction of ergodic time series

TL;DR

This paper introduces a simple randomized algorithm for predicting binary sequences that performs near-optimally for stationary ergodic processes, including Markov processes, and extends to predictions with side information.

Contribution

The paper proposes a new randomized prediction algorithm that guarantees almost sure convergence to the optimal predictor for ergodic sequences, without prior knowledge of process order.

Findings

Converges almost surely to the Bayes predictor for stationary ergodic sequences.

Performs near-optimally on Markov processes without knowing their order.

Effective prediction with side information included.

Abstract

We present a simple randomized procedure for the prediction of a binary sequence. The algorithm uses ideas from recent developments of the theory of the prediction of individual sequences. We show that if the sequence is a realization of a stationary and ergodic random process then the average number of mistakes converges, almost surely, to that of the optimum, given by the Bayes predictor. The desirable finite-sample properties of the predictor are illustrated by its performance for Markov processes. In such cases the predictor exhibits near optimal behavior even without knowing the order of the Markov process. Prediction with side information is also considered.