Algorithmic learning of probability distributions from random data in the limit

TL;DR

This paper investigates the limits of algorithmic learning of probability distributions from data, showing computability results and characterizing oracles that enable explanatory learning of computable measures.

Contribution

It characterizes the oracles that can compute explanatory learners for computable probability measures, extending previous results and analyzing various learning notions.

Findings

Existence of a computable partial learner for computable measures

No computable learner exists for all computable measures

High oracles are necessary for explanatory learning of continuous measures

Abstract

We study the problem of identifying a probability distribution for some given randomly sampled data in the limit, in the context of algorithmic learning theory as proposed recently by Vinanyi and Chater. We show that there exists a computable partial learner for the computable probability measures, while by Bienvenu, Monin and Shen it is known that there is no computable learner for the computable probability measures. Our main result is the characterization of the oracles that compute explanatory learners for the computable (continuous) probability measures as the high oracles. This provides an analogue of a well-known result of Adleman and Blum in the context of learning computable probability distributions. We also discuss related learning notions such as behaviorally correct learning and orther variations of explanatory learning, in the context of learning probability distributions from data.