Distributional Adversarial Loss

TL;DR

This paper introduces a novel distributional adversarial loss framework that considers perturbation sets as distributions, providing theoretical bounds and unifying randomized smoothing with robust learning, along with empirical derandomization techniques.

Contribution

It proposes a new distributional adversarial loss concept, derives sample complexity bounds, and unifies randomized smoothing with robust learning in the PAC setting.

Findings

Sample complexity bounds for distributional adversarial loss.

Unified framework for randomized smoothing and robust learning.

Effective derandomization technique preserving robustness.

Abstract

We initiate the study of a new notion of adversarial loss which we call distributional adversarial loss. In this notion, we assume for each original example, the allowed adversarial perturbation set is a family of distributions, and the adversarial loss over each example is the maximum loss over all the associated distributions. The goal is to minimize the overall adversarial loss. We show sample complexity bounds in the PAC-learning setting for our notion of adversarial loss. Our notion of adversarial loss contrasts the prior work on robust learning that considers a set of points, not distributions, as the perturbation set of each clean example. As an application of our approach, we show how to unify the two lines of work on randomized smoothing and robust learning in the PAC-learning setting and derive sample complexity bounds for randomized smoothing methods. Furthermore, we investigate the role of randomness in achieving robustness against adversarial attacks. We show a general derandomization technique that preserves the extent of a randomized classifier's robustness against adversarial attacks and show its effectiveness empirically.