The Sample Complexity of Replicable Realizable PAC Learning

TL;DR

This paper investigates the sample complexity of replicable PAC learning for realizable problems, establishing near-tight bounds that depend on the size of the hypothesis class using novel graph-theoretic techniques.

Contribution

It introduces a new lower bound on sample complexity with a $( ext{log}|H|)^{3/2}$ dependence and employs innovative spectral graph analysis methods.

Findings

Established a near-tight lower bound on sample complexity.

Developed novel techniques involving Cayley graphs and spectral analysis.

Provided an almost matching upper bound for the constructed hard instance.

Abstract

In this paper, we consider the problem of replicable realizable PAC learning. We construct a particularly hard learning problem and show a sample complexity lower bound with a close to $(\log|H|)^{3/2}$ dependence on the size of the hypothesis class $H$. Our proof uses several novel techniques and works by defining a particular Cayley graph associated with $H$ and analyzing a suitable random walk on this graph by examining the spectral properties of its adjacency matrix. Furthermore, we show an almost matching upper bound for the lower bound instance, meaning if a stronger lower bound exists, one would have to consider a different instance of the problem.