Computable learning of natural hypothesis classes

TL;DR

This paper explores the boundaries of computable learning in natural hypothesis classes, showing that under mild conditions, learnability implies computable learnability, thus challenging previous counterexamples that were unnatural.

Contribution

It proves that natural hypothesis classes that are learnable under mild assumptions are necessarily computably learnable, bridging a gap in computable learning theory.

Findings

Counterexamples to computable learnability are unnatural.

Natural hypothesis classes are computably learnable if they are learnable.

Using on-a-cone machinery from computability theory to establish results.

Abstract

This paper is about the recent notion of computably probably approximately correct learning, which lies between the statistical learning theory where there is no computational requirement on the learner and efficient PAC where the learner must be polynomially bounded. Examples have recently been given of hypothesis classes which are PAC learnable but not computably PAC learnable, but these hypothesis classes are unnatural or non-canonical in the sense that they depend on a numbering of proofs, formulas, or programs. We use the on-a-cone machinery from computability theory to prove that, under mild assumptions such as that the hypothesis class can be computably listable, any natural hypothesis class which is learnable must be computably learnable. Thus the counterexamples given previously are necessarily unnatural.