Efficient Optimal PAC Learning

TL;DR

This paper explores the computational tradeoffs of optimal PAC learners in binary classification, proposing an alternative approach that balances efficiency and optimality by modifying the reliance on empirical risk minimization.

Contribution

It introduces a new optimal PAC learner that offers a different computational tradeoff compared to existing methods, reducing dependence on empirical risk minimization.

Findings

Existence of an optimal PAC learner with alternative computational tradeoffs.

Demonstrates the link between PAC learning optimality and empirical risk minimization.

Provides insights into balancing computational cost and learning optimality.

Abstract

Recent advances in the binary classification setting by Hanneke [2016b] and Larsen [2023] have resulted in optimal PAC learners. These learners leverage, respectively, a clever deterministic subsampling scheme and the classic heuristic of bagging Breiman [1996]. Both optimal PAC learners use, as a subroutine, the natural algorithm of empirical risk minimization. Consequently, the computational cost of these optimal PAC learners is tied to that of the empirical risk minimizer algorithm. In this work, we seek to provide an alternative perspective on the computational cost imposed by the link to the empirical risk minimizer algorithm. To this end, we show the existence of an optimal PAC learner, which offers a different tradeoff in terms of the computational cost induced by the empirical risk minimizer.