SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation

TL;DR

SPADE is a spectral method that evaluates and enhances the adversarial robustness of machine learning models by analyzing input/output data manifolds and identifying vulnerable samples, leading to improved training strategies.

Contribution

Introduces SPADE, a spectral approach utilizing graph theory and eigenvector analysis to evaluate and improve model robustness against adversarial attacks.

Findings

SPADE provides an upper bound on the Lipschitz constant for robustness evaluation.

Spectral embedding identifies highly vulnerable data samples.

Empirical results show improved robustness on MNIST and CIFAR-10 datasets.

Abstract

A black-box spectral method is introduced for evaluating the adversarial robustness of a given machine learning (ML) model. Our approach, named SPADE, exploits bijective distance mapping between the input/output graphs constructed for approximating the manifolds corresponding to the input/output data. By leveraging the generalized Courant-Fischer theorem, we propose a SPADE score for evaluating the adversarial robustness of a given model, which is proved to be an upper bound of the best Lipschitz constant under the manifold setting. To reveal the most non-robust data samples highly vulnerable to adversarial attacks, we develop a spectral graph embedding procedure leveraging dominant generalized eigenvectors. This embedding step allows assigning each data sample a robustness score that can be further harnessed for more effective adversarial training. Our experiments show the proposed SPADE method leads to promising empirical results for neural network models that are adversarially trained with the MNIST and CIFAR-10 data sets.