A Monte Carlo Algorithm for Universally Optimal Bayesian Sequence Prediction and Planning

TL;DR

This paper explores the design of rational decision-making agents using a Monte Carlo algorithm to approximate optimal Bayesian sequence prediction and planning within resource-bounded computational models.

Contribution

It introduces a Monte Carlo method for approximating universally optimal Bayesian sequence prediction and planning in resource-bounded settings, advancing AI decision-making theory.

Findings

Monte Carlo algorithm effectively approximates Bayesian prediction and planning.

Resource limitations influence the architecture of learning systems.

Insights into implementing near-optimal decision makers in practical AI systems.

Abstract

The aim of this work is to address the question of whether we can in principle design rational decision-making agents or artificial intelligences embedded in computable physics such that their decisions are optimal in reasonable mathematical senses. Recent developments in rare event probability estimation, recursive bayesian inference, neural networks, and probabilistic planning are sufficient to explicitly approximate reinforcement learners of the AIXI style with non-trivial model classes (here, the class of resource-bounded Turing machines). Consideration of the effects of resource limitations in a concrete implementation leads to insights about possible architectures for learning systems using optimal decision makers as components.