Algorithmic Complexity Bounds on Future Prediction Errors

TL;DR

This paper establishes bounds on future prediction errors for stochastic sequences using a new form of algorithmic complexity that accounts for past observations and randomness deficiency, extending previous universal prediction results.

Contribution

It introduces a monotone variant of algorithmic complexity conditioned on observed data to bound future prediction errors, generalizing Solomonoff's universal predictor bounds.

Findings

Bound on future prediction loss using new complexity measure

Monotonicity property of the complexity with respect to data extension

Potential extensions to Bayesian models and classification tasks

Abstract

We bound the future loss when predicting any (computably) stochastic sequence online. Solomonoff finitely bounded the total deviation of his universal predictor $M$ from the true distribution $mu$ by the algorithmic complexity of $mu$. Here we assume we are at a time $t>1$ and already observed $x=x_1...x_t$. We bound the future prediction performance on $x_{t+1}x_{t+2}...$ by a new variant of algorithmic complexity of $mu$ given $x$, plus the complexity of the randomness deficiency of $x$. The new complexity is monotone in its condition in the sense that this complexity can only decrease if the condition is prolonged. We also briefly discuss potential generalizations to Bayesian model classes and to classification problems.