A Characterization of Semi-Supervised Adversarially-Robust PAC Learnability

TL;DR

This paper analyzes the sample complexity of semi-supervised adversarially robust PAC learning, demonstrating that unlabeled data significantly reduces the number of labeled examples needed for robust learning.

Contribution

It provides nearly matching bounds on labeled sample complexity in semi-supervised robust PAC learning, highlighting the benefits over fully-supervised methods.

Findings

Unlabeled data can drastically reduce labeled sample requirements.

A different complexity measure characterizes labeled sample complexity.

Semi-supervised robust learning outperforms supervised learning in worst-case scenarios.

Abstract

We study the problem of learning an adversarially robust predictor to test time attacks in the semi-supervised PAC model. We address the question of how many labeled and unlabeled examples are required to ensure learning. We show that having enough unlabeled data (the size of a labeled sample that a fully-supervised method would require), the labeled sample complexity can be arbitrarily smaller compared to previous works, and is sharply characterized by a different complexity measure. We prove nearly matching upper and lower bounds on this sample complexity. This shows that there is a significant benefit in semi-supervised robust learning even in the worst-case distribution-free model, and establishes a gap between the supervised and semi-supervised label complexities which is known not to hold in standard non-robust PAC learning.