On the Computability of Robust PAC Learning

TL;DR

This paper explores the computability aspects of adversarially robust PAC learning, introducing a new framework and dimension, revealing that learnability does not always follow from component properties and that some aspects are surprisingly non-computable.

Contribution

It introduces the robust computable PAC (robust CPAC) learning framework, analyzes its properties, and proposes the computable robust shattering dimension as a new measure for understanding learnability.

Findings

Learnability in robust CPAC is not implied by its components.

Robust CPAC learnability does not require the robust loss to be computably evaluable.

Finiteness of the computable robust shattering dimension is necessary but not sufficient for learnability.

Abstract

We initiate the study of computability requirements for adversarially robust learning. Adversarially robust PAC-type learnability is by now an established field of research. However, the effects of computability requirements in PAC-type frameworks are only just starting to emerge. We introduce the problem of robust computable PAC (robust CPAC) learning and provide some simple sufficient conditions for this. We then show that learnability in this setup is not implied by the combination of its components: classes that are both CPAC and robustly PAC learnable are not necessarily robustly CPAC learnable. Furthermore, we show that the novel framework exhibits some surprising effects: for robust CPAC learnability it is not required that the robust loss is computably evaluable! Towards understanding characterizing properties, we introduce a novel dimension, the computable robust shattering dimension. We prove that its finiteness is necessary, but not sufficient for robust CPAC learnability. This might yield novel insights for the corresponding phenomenon in the context of robust PAC learnability, where insufficiency of the robust shattering dimension for learnability has been conjectured, but so far a resolution has remained elusive.