On the equivalence of Occam algorithms

TL;DR

This paper extends the theoretical understanding of Occam algorithms by showing that the partial converse of their learnability equivalence applies to algorithms with complexity independent of confidence parameters, strengthening the foundational link between Occam algorithms and PAC learnability.

Contribution

It demonstrates that the partial converse of the equivalence between Occam algorithms and PAC learnability holds for complexity measures independent of confidence parameters.

Findings

The partial converse applies to $ ext{delta}$-independent complexities.

Provides theoretical justification for algorithms based on the partial converse.

Strengthens the link between Occam algorithms and PAC learnability.

Abstract

Blumer et al. (1987, 1989) showed that any concept class that is learnable by Occam algorithms is PAC learnable. Board and Pitt (1990) showed a partial converse of this theorem: for concept classes that are closed under exception lists, any class that is PAC learnable is learnable by an Occam algorithm. However, their Occam algorithm outputs a hypothesis whose complexity is $\delta$-dependent, which is an important limitation. In this paper, we show that their partial converse applies to Occam algorithms with $\delta$-independent complexities as well. Thus, we provide a posteriori justification of various theoretical results and algorithm design methods which use the partial converse as a basis for their work.