VC Classes are Adversarially Robustly Learnable, but Only Improperly

TL;DR

This paper demonstrates that all finite VC dimension hypothesis classes are learnable in an adversarially robust manner using improper learning rules, but proper rules are insufficient for some classes.

Contribution

It proves that improper learning rules enable adversarially robust PAC learning for all finite VC classes, highlighting the necessity of improper learning.

Findings

Finite VC classes are robustly PAC learnable with improper rules.

Proper learning rules are not sufficient for some finite VC classes.

Improper learning is necessary for adversarial robustness in certain classes.

Abstract

We study the question of learning an adversarially robust predictor. We show that any hypothesis class $\mathcal{H}$ with finite VC dimension is robustly PAC learnable with an improper learning rule. The requirement of being improper is necessary as we exhibit examples of hypothesis classes $\mathcal{H}$ with finite VC dimension that are not robustly PAC learnable with any proper learning rule.