MAD: Meta Adversarial Defense Benchmark

TL;DR

This paper introduces MAD, a benchmark for adversarial defense using meta-learning, featuring datasets, evaluation protocols, and a baseline algorithm that improves robustness and generalization against unseen attacks.

Contribution

The paper proposes a novel MAD benchmark with datasets and evaluation protocols, along with a meta-learning based adversarial training algorithm that enhances robustness and generalization.

Findings

Meta-AT outperforms state-of-the-art methods in robustness.

Meta-AT maintains high accuracy on clean samples.

MAD benchmark facilitates transferability and few-shot learning in adversarial defense.

Abstract

Adversarial training (AT) is a prominent technique employed by deep learning models to defend against adversarial attacks, and to some extent, enhance model robustness. However, there are three main drawbacks of the existing AT-based defense methods: expensive computational cost, low generalization ability, and the dilemma between the original model and the defense model. To this end, we propose a novel benchmark called meta adversarial defense (MAD). The MAD benchmark consists of two MAD datasets, along with a MAD evaluation protocol. The two large-scale MAD datasets were generated through experiments using 30 kinds of attacks on MNIST and CIFAR-10 datasets. In addition, we introduce a meta-learning based adversarial training (Meta-AT) algorithm as the baseline, which features high robustness to unseen adversarial attacks through few-shot learning. Experimental results demonstrate the effectiveness of our Meta-AT algorithm compared to the state-of-the-art methods. Furthermore, the model after Meta-AT maintains a relatively high clean-samples classification accuracy (CCA). It is worth noting that Meta-AT addresses all three aforementioned limitations, leading to substantial improvements. This benchmark ultimately achieved breakthroughs in investigating the transferability of adversarial defense methods to new attacks and the ability to learn from a limited number of adversarial examples. Our codes and attacked datasets address will be available at https://github.com/PXX1110/Meta_AT.