A Dilemma for Solomonoff Prediction

TL;DR

This paper discusses the inherent issues in Solomonoff prediction, highlighting the tension between its dependence on universal Turing machines and the non-computability of its prior, despite existing responses and approximations.

Contribution

It analyzes the conflicting responses to Solomonoff prediction's problems, emphasizing the tension between convergence and computability of approximations.

Findings

Different Solomonoff priors converge with more data

Computable approximations do not always converge

There is a fundamental tension between convergence and computability

Abstract

The framework of Solomonoff prediction assigns prior probability to hypotheses inversely proportional to their Kolmogorov complexity. There are two well-known problems. First, the Solomonoff prior is relative to a choice of Universal Turing machine. Second, the Solomonoff prior is not computable. However, there are responses to both problems. Different Solomonoff priors converge with more and more data. Further, there are computable approximations to the Solomonoff prior. I argue that there is a tension between these two responses. This is because computable approximations to Solomonoff prediction do not always converge.