Recursively Enumerably Representable Classes and Computable Versions of the Fundamental Theorem of Statistical Learning

TL;DR

This paper explores the relationship between computable PAC learning and recursively enumerable classes, revealing how effective VC-dimensions vary and establishing conditions under which classes are learnable or identifiable.

Contribution

It introduces new connections between CPAC learning and RER classes, showing when effective VC-dimensions align and characterizing learnability through RER class containment.

Findings

Effective VC-dimensions can vary widely for RER classes.

CPAC learnability aligns with RER class containment under certain conditions.

RER classes can be agnostically learned with relaxed CPAC notions.

Abstract

We study computable probably approximately correct (CPAC) learning, where learners are required to be computable functions. It had been previously observed that the Fundamental Theorem of Statistical Learning, which characterizes PAC learnability by finiteness of the Vapnik-Chervonenkis (VC-)dimension, no longer holds in this framework. Recent works recovered analogs of the Fundamental Theorem in the computable setting, for instance by introducing an effective VC-dimension. Guided by this, we investigate the connection between CPAC learning and recursively enumerable representable (RER) classes, whose members can be algorithmically listed. Our results show that the effective VC-dimensions can take arbitrary values above the traditional one, even for RER classes, which creates a whole family of (non-)examples for various notions of CPAC learning. Yet the two dimensions coincide for classes satisfying sufficiently strong notions of CPAC learning. We then observe that CPAC learnability can also be characterized via containment of RER classes that realize the same samples. Furthermore, it is shown that CPAC learnable classes satisfying a unique identification property are necessarily RER. Finally, we establish that agnostic learnability can be guaranteed for RER classes, by considering the relaxed notion of nonuniform CPAC learning.