Adversarially Robust Learning: A Generic Minimax Optimal Learner and Characterization

TL;DR

This paper introduces a minimax optimal adversarially robust learning algorithm, utilizing a novel global perspective and a new complexity measure, resolving key open problems and improving understanding of robust learnability.

Contribution

It presents a new minimax optimal learner for adversarial robustness, introduces the global one-inclusion graph, and characterizes the predictor classes that are robustly learnable.

Findings

The proposed learner is minimax optimal for adversarial robustness.

The global one-inclusion graph generalizes classical models and aids analysis.

A new dimension characterizes robust learnability of predictor classes.

Abstract

We present a minimax optimal learner for the problem of learning predictors robust to adversarial examples at test-time. Interestingly, we find that this requires new algorithmic ideas and approaches to adversarially robust learning. In particular, we show, in a strong negative sense, the suboptimality of the robust learner proposed by Montasser, Hanneke, and Srebro (2019) and a broader family of learners we identify as local learners. Our results are enabled by adopting a global perspective, specifically, through a key technical contribution: the global one-inclusion graph, which may be of independent interest, that generalizes the classical one-inclusion graph due to Haussler, Littlestone, and Warmuth (1994). Finally, as a byproduct, we identify a dimension characterizing qualitatively and quantitatively what classes of predictors $\mathcal{H}$ are robustly learnable. This resolves an open problem due to Montasser et al. (2019), and closes a (potentially) infinite gap between the established upper and lower bounds on the sample complexity of adversarially robust learning.