On the Foundations of Universal Sequence Prediction

TL;DR

This paper discusses Solomonoff's universal sequence prediction model, highlighting its theoretical advantages such as fast convergence, invariance, and effectiveness in non-computable environments, addressing philosophical issues of traditional Bayesian methods.

Contribution

It provides a comprehensive analysis of Solomonoff's model, emphasizing its desirable properties and its solutions to philosophical problems in Bayesian sequence prediction.

Findings

Fast convergence and strong bounds established

No zero prior problem, enabling hypothesis confirmation

Performs well even in non-computable environments

Abstract

Solomonoff completed the Bayesian framework by providing a rigorous, unique, formal, and universal choice for the model class and the prior. We discuss in breadth how and in which sense universal (non-i.i.d.) sequence prediction solves various (philosophical) problems of traditional Bayesian sequence prediction. We show that Solomonoff's model possesses many desirable properties: Fast convergence and strong bounds, and in contrast to most classical continuous prior densities has no zero p(oste)rior problem, i.e. can confirm universal hypotheses, is reparametrization and regrouping invariant, and avoids the old-evidence and updating problem. It even performs well (actually better) in non-computable environments.