Towards the Desirable Decision Boundary by Moderate-Margin Adversarial Training

TL;DR

This paper introduces Moderate-Margin Adversarial Training (MMAT), a novel method that balances robustness and natural accuracy by learning a moderate-inclusive decision boundary using finer adversarial examples and teacher guidance.

Contribution

The paper proposes MMAT, a new adversarial training scheme that mitigates the cross-over problem and improves the robustness-accuracy trade-off with teacher guidance.

Findings

Achieves state-of-the-art robustness and natural accuracy on SVHN.

Effectively balances robustness and natural accuracy.

Utilizes finer-grained adversarial examples and teacher logits.

Abstract

Adversarial training, as one of the most effective defense methods against adversarial attacks, tends to learn an inclusive decision boundary to increase the robustness of deep learning models. However, due to the large and unnecessary increase in the margin along adversarial directions, adversarial training causes heavy cross-over between natural examples and adversarial examples, which is not conducive to balancing the trade-off between robustness and natural accuracy. In this paper, we propose a novel adversarial training scheme to achieve a better trade-off between robustness and natural accuracy. It aims to learn a moderate-inclusive decision boundary, which means that the margins of natural examples under the decision boundary are moderate. We call this scheme Moderate-Margin Adversarial Training (MMAT), which generates finer-grained adversarial examples to mitigate the cross-over problem. We also take advantage of logits from a teacher model that has been well-trained to guide the learning of our model. Finally, MMAT achieves high natural accuracy and robustness under both black-box and white-box attacks. On SVHN, for example, state-of-the-art robustness and natural accuracy are achieved.