On Universal Prediction and Bayesian Confirmation

TL;DR

This paper discusses Solomonoff's universal sequence prediction model, which enhances Bayesian methods by providing a rigorous, invariant, and universally applicable prior, addressing limitations of traditional Bayesian approaches in complex and non-computable environments.

Contribution

It introduces and analyzes Solomonoff's universal model, demonstrating its theoretical advantages and robustness over classical Bayesian methods in various complex scenarios.

Findings

Provides strong theoretical bounds for universal prediction

Shows Solomonoff's model can confirm hypotheses without zero prior issues

Demonstrates robustness in non-computable environments

Abstract

The Bayesian framework is a well-studied and successful framework for inductive reasoning, which includes hypothesis testing and confirmation, parameter estimation, sequence prediction, classification, and regression. But standard statistical guidelines for choosing the model class and prior are not always available or fail, in particular in complex situations. Solomonoff completed the Bayesian framework by providing a rigorous, unique, formal, and universal choice for the model class and the prior. We discuss in breadth how and in which sense universal (non-i.i.d.) sequence prediction solves various (philosophical) problems of traditional Bayesian sequence prediction. We show that Solomonoff's model possesses many desirable properties: Strong total and weak instantaneous bounds, and in contrast to most classical continuous prior densities has no zero p(oste)rior problem, i.e. can confirm universal hypotheses, is reparametrization and regrouping invariant, and avoids the old-evidence and updating problem. It even performs well (actually better) in non-computable environments.