New Error Bounds for Solomonoff Prediction

TL;DR

This paper establishes new theoretical error bounds for Solomonoff sequence prediction, showing its optimality and finite error properties for computable distributions, and compares it with informed and deterministic schemes.

Contribution

It introduces novel relations between Solomonoff prediction and other schemes, proving its near-optimality and finite error bounds for computable distributions.

Findings

Finite error in Solomonoff prediction for computable distributions

Deterministic Solomonoff prediction is nearly optimal

New bounds complement existing Solomonoff bounds

Abstract

Solomonoff sequence prediction is a scheme to predict digits of binary strings without knowing the underlying probability distribution. We call a prediction scheme informed when it knows the true probability distribution of the sequence. Several new relations between universal Solomonoff sequence prediction and informed prediction and general probabilistic prediction schemes will be proved. Among others, they show that the number of errors in Solomonoff prediction is finite for computable distributions, if finite in the informed case. Deterministic variants will also be studied. The most interesting result is that the deterministic variant of Solomonoff prediction is optimal compared to any other probabilistic or deterministic prediction scheme apart from additive square root corrections only. This makes it well suited even for difficult prediction problems, where it does not suffice when the number of errors is minimal to within some factor greater than one. Solomonoff's original bound and the ones presented here complement each other in a useful way.