Equivalences between learning of data and probability distributions, and their applications

TL;DR

This paper establishes a fundamental connection between the learnability of data sequences and probability distributions, enabling transfer of classical results to probabilistic learning scenarios.

Contribution

It proves an equivalence between the learnability of certain classes of probability measures and their real parameters, facilitating new insights and results in probabilistic learning theory.

Findings

Equivalence between measure and real parameter learnability

Transfer of classical algorithmic results to measure learning

New results on EX and BC learnability of measure classes

Abstract

Algorithmic learning theory traditionally studies the learnability of effective infinite binary sequences (reals), while recent work by [Vitanyi and Chater, 2017] and [Bienvenu et al., 2014] has adapted this framework to the study of learnability of effective probability distributions from random data. We prove that for certain families of probability measures that are parametrized by reals, learnability of a subclass of probability measures is equivalent to learnability of the class of the corresponding real parameters. This equivalence allows to transfer results from classical algorithmic theory to learning theory of probability measures. We present a number of such applications, providing many new results regarding EX and BC learnability of classes of measures, thus drawing parallels between the two learning theories.