Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier

TL;DR

This paper investigates the conditions under which convex surrogate losses lead to statistically consistent adversarial classifiers, linking consistency to the uniqueness of adversarial Bayes classifiers.

Contribution

It establishes a theoretical connection between the consistency of convex surrogate losses and the uniqueness of adversarial Bayes classifiers under certain assumptions.

Findings

Convex surrogate losses are not always statistically consistent in adversarial settings.

Consistency is achieved iff the adversarial Bayes classifier is unique.

Provides conditions under which adversarial surrogate risk minimization aligns with true adversarial classification error.

Abstract

Minimizing an adversarial surrogate risk is a common technique for learning robust classifiers. Prior work showed that convex surrogate losses are not statistically consistent in the adversarial context -- or in other words, a minimizing sequence of the adversarial surrogate risk will not necessarily minimize the adversarial classification error. We connect the consistency of adversarial surrogate losses to properties of minimizers to the adversarial classification risk, known as adversarial Bayes classifiers. Specifically, under reasonable distributional assumptions, a convex surrogate loss is statistically consistent for adversarial learning iff the adversarial Bayes classifier satisfies a certain notion of uniqueness.