Guided Interpolation for Adversarial Training

TL;DR

This paper introduces the Guided Interpolation Framework (GIF) that improves adversarial training by generating more attackable data, thereby enhancing robustness and reducing linearity between classes.

Contribution

The paper proposes a novel GIF method that uses previous epoch's meta information to guide data interpolation, improving adversarial robustness over vanilla mixup.

Findings

GIF increases attackable data ratio during training

GIF enhances robustness across different methods and datasets

GIF reduces linear behavior between classes

Abstract

To enhance adversarial robustness, adversarial training learns deep neural networks on the adversarial variants generated by their natural data. However, as the training progresses, the training data becomes less and less attackable, undermining the robustness enhancement. A straightforward remedy is to incorporate more training data, but sometimes incurring an unaffordable cost. In this paper, to mitigate this issue, we propose the guided interpolation framework (GIF): in each epoch, the GIF employs the previous epoch's meta information to guide the data's interpolation. Compared with the vanilla mixup, the GIF can provide a higher ratio of attackable data, which is beneficial to the robustness enhancement; it meanwhile mitigates the model's linear behavior between classes, where the linear behavior is favorable to generalization but not to the robustness. As a result, the GIF encourages the model to predict invariantly in the cluster of each class. Experiments demonstrate that the GIF can indeed enhance adversarial robustness on various adversarial training methods and various datasets.