Monotone Conditional Complexity Bounds on Future Prediction Errors

TL;DR

This paper introduces a new monotone conditional complexity measure to bound future prediction errors in stochastic sequences, improving understanding of how past observations influence future predictions.

Contribution

It proposes a novel monotone conditional complexity framework that tightens bounds on future prediction errors based on observed data and randomness deficiency.

Findings

Bound on future loss using the new complexity measure

Monotonicity property of the complexity in the condition

Potential extensions to Bayesian models and classification

Abstract

We bound the future loss when predicting any (computably) stochastic sequence online. Solomonoff finitely bounded the total deviation of his universal predictor M from the true distribution m by the algorithmic complexity of m. Here we assume we are at a time t>1 and already observed x=x_1...x_t. We bound the future prediction performance on x_{t+1}x_{t+2}... by a new variant of algorithmic complexity of m given x, plus the complexity of the randomness deficiency of x. The new complexity is monotone in its condition in the sense that this complexity can only decrease if the condition is prolonged. We also briefly discuss potential generalizations to Bayesian model classes and to classification problems.