Mutual Adversarial Training: Learning together is better than going alone

TL;DR

This paper introduces mutual adversarial training (MAT), a collaborative learning approach where multiple models learn together to enhance robustness against adversarial attacks, outperforming existing methods on CIFAR datasets.

Contribution

The paper proposes a novel mutual adversarial training method enabling models to learn collaboratively and share adversarial knowledge, improving robustness beyond static teacher-student frameworks.

Findings

MAT improves robustness by ~8% over vanilla adversarial training under PGD-100 attacks.

MAT mitigates robustness trade-offs across different perturbation types, gaining up to 13.1%.

Extensive experiments validate MAT's effectiveness on CIFAR-10 and CIFAR-100 datasets.

Abstract

Recent studies have shown that robustness to adversarial attacks can be transferred across networks. In other words, we can make a weak model more robust with the help of a strong teacher model. We ask if instead of learning from a static teacher, can models "learn together" and "teach each other" to achieve better robustness? In this paper, we study how interactions among models affect robustness via knowledge distillation. We propose mutual adversarial training (MAT), in which multiple models are trained together and share the knowledge of adversarial examples to achieve improved robustness. MAT allows robust models to explore a larger space of adversarial samples, and find more robust feature spaces and decision boundaries. Through extensive experiments on CIFAR-10 and CIFAR-100, we demonstrate that MAT can effectively improve model robustness and outperform state-of-the-art methods under white-box attacks, bringing $\sim$8% accuracy gain to vanilla adversarial training (AT) under PGD-100 attacks. In addition, we show that MAT can also mitigate the robustness trade-off among different perturbation types, bringing as much as 13.1% accuracy gain to AT baselines against the union of $l_\infty$, $l_2$ and $l_1$ attacks. These results show the superiority of the proposed method and demonstrate that collaborative learning is an effective strategy for designing robust models.