Achieving Adversarial Robustness Requires An Active Teacher

TL;DR

This paper proposes a new framework for understanding adversarial robustness by decoupling data and label generation, emphasizing the importance of an active teacher in achieving robustness.

Contribution

It introduces a novel perspective that robustness depends on the teacher actively providing information, challenging traditional passive training assumptions.

Findings

Active teachers improve robustness efficiency.

Decoupling data and label generation clarifies adversarial example origins.

Theoretical and numerical evidence supports active teaching for robustness.

Abstract

A new understanding of adversarial examples and adversarial robustness is proposed by decoupling the data generator and the label generator (which we call the teacher). In our framework, adversarial robustness is a conditional concept---the student model is not absolutely robust, but robust with respect to the teacher. Based on the new understanding, we claim that adversarial examples exist because the student cannot obtain sufficient information of the teacher from the training data. Various ways of achieving robustness is compared. Theoretical and numerical evidence shows that to efficiently attain robustness, a teacher that actively provides its information to the student may be necessary.