Evaluating Adversarial Robustness with Expected Viable Performance

TL;DR

This paper proposes a new metric for assessing classifier robustness against adversarial attacks by measuring expected functionality over possible perturbations, providing a domain-general robustness evaluation framework.

Contribution

It introduces an expected viability-based robustness metric that accounts for adversarial perturbations, offering a novel, domain-agnostic approach to robustness evaluation.

Findings

The metric quantifies robustness in terms of expected classifier functionality.

It highlights the importance of considering the distribution of adversarial perturbations.

Provides a new perspective on robustness beyond traditional accuracy measures.

Abstract

We introduce a metric for evaluating the robustness of a classifier, with particular attention to adversarial perturbations, in terms of expected functionality with respect to possible adversarial perturbations. A classifier is assumed to be non-functional (that is, has a functionality of zero) with respect to a perturbation bound if a conventional measure of performance, such as classification accuracy, is less than a minimally viable threshold when the classifier is tested on examples from that perturbation bound. Defining robustness in terms of an expected value is motivated by a domain general approach to robustness quantification.