Sequence Prediction based on Monotone Complexity

TL;DR

This paper investigates the use of monotone Kolmogorov complexity for sequence prediction, comparing it to Solomonoff's prior, and analyzes its convergence properties in deterministic and probabilistic environments.

Contribution

It provides theoretical analysis of the prediction performance of monotone complexity-based models, highlighting their convergence behavior and limitations.

Findings

Posterior and losses converge in deterministic computable environments.

Rapid convergence can only be shown on-sequence, off-sequence behavior remains unclear.

In probabilistic environments, neither the posterior nor the losses generally converge.

Abstract

This paper studies sequence prediction based on the monotone Kolmogorov complexity Km=-log m, i.e. based on universal deterministic/one-part MDL. m is extremely close to Solomonoff's prior M, the latter being an excellent predictor in deterministic as well as probabilistic environments, where performance is measured in terms of convergence of posteriors or losses. Despite this closeness to M, it is difficult to assess the prediction quality of m, since little is known about the closeness of their posteriors, which are the important quantities for prediction. We show that for deterministic computable environments, the "posterior" and losses of m converge, but rapid convergence could only be shown on-sequence; the off-sequence behavior is unclear. In probabilistic environments, neither the posterior nor the losses converge, in general.